# Goliash

> What runs where, on which version, in which environment: self-hosted, open-source version tracking for Kubernetes, Amazon ECS, AWS Lambda, Nomad, Docker Swarm and Compose. https://goliash.dev

# Getting started
Source: /getting-started/

> Run Goliash with Docker in a minute, load example data, and add your first environment, agent and target.

Just looking? The [live demo](https://demo.goliash.dev) needs no install. Or one command runs a throwaway server with three weeks of example data and prints a link that signs you
in; nothing is kept when you stop it:

```sh
docker run --rm -p 8080:8080 ghcr.io/pipozzz/goliash try
```

The rest of this page starts a server to keep with Docker Compose, shows example data, and connects your first
orchestrator. It takes about five minutes.

## 1. Start the server

```sh
curl -fsSLO https://raw.githubusercontent.com/pipozzz/goliash/main/docker-compose.yml
docker compose up -d
docker compose logs goliash | grep link=        # open it to create your account
```

The server listens on [http://localhost:8080](http://localhost:8080) and keeps its SQLite database in a Docker volume.

## 2. Create your account

Open [http://localhost:8080](http://localhost:8080): for the first 15 minutes after the start, while nobody has an
account, the sign-in page offers **Create the first account**. Enter your e-mail and a password: you are the owner of
the organization and signed in. Later the offer is gone from the page and the one-time **setup link** is only in the
server log (the last command above shows it); it works once, for 24 hours, and restarting the server writes a new one.
Invite the others from **Settings → Users**.

## 3. Look around with example data (optional)

The empty overview shows **Get started** with the next step highlighted. Below it, **Explore with example data**
opens a separate *Example* workspace with three weeks of realistic history: four targets, seven services, rollouts,
drift and new upstream releases. Your own workspace stays empty and clean; switch back in the workspace menu at the
top and delete *Example* under **Settings → Workspaces** when you are done.

To put the example data into the current workspace instead, run `docker compose exec goliash goliash demo`; its
targets are named `demo-*`, so they are easy to tell apart from real ones.

## 4. Connect what you run

There are two ways to collect a target. With **Map what it finds to services automatically** left on (it is
offered while the workspace is empty), the matrix fills by itself within a minute of the first report; otherwise
the workloads wait in the inbox for you to name their services.

An agent runs next to the orchestrator and sends snapshots to the server over HTTPS. Use it when the server
cannot reach the orchestrator, or when credentials must stay in that network.

1. Open **Connect** (on the matrix of a new installation, or **Settings → Agents and targets → Connect a cluster or host**) and click
   where the agent will run: **Kubernetes**, **Docker or Docker Compose host**, **Nomad**, or **AWS, without a
   server** (ECS Fargate). Above the tiles, pick the environment (production by default) or name a new one; what the
   agent finds maps to services by itself while the workspace has none. For a code for many hosts or a name, use the
   links under the tiles; Swarm, Linux servers and Compose files from Git are under **More
   ways**.
2. Run the one command the page shows where the agent runs: Helm for Kubernetes, files in `~/goliash-agent` for
   Docker, a job for Nomad, a CloudFormation stack for AWS. The enrollment code is already in it and shown only once.
3. The agent finds by itself what it can read there: workloads, Compose projects, the host's OS and packages, and on
   EC2 or EKS the AWS services its role may read. It looks again every 15 minutes and adds what turns up.
4. Keep the page open: it shows when the agent connects and what its first report found, then takes you to the
   matrix or the inbox.

The manifests behind those commands are under [Install](/install/).

The server can collect targets itself, with the same read-only collectors: in **Connect**, pick **This server**
instead of an agent. Compose files from Git need nothing else. To watch the Docker host the quickstart runs on,
start its read-only socket proxy and connect a Docker host with the address `tcp://socket-proxy:2375`, or from the
command line:

```sh
docker compose --profile watch-host up -d               # adds a read-only docker-socket-proxy
docker compose exec goliash goliash env create -name prod -position 30
docker compose exec goliash goliash target create -env prod -platform docker -name this-host \
  -settings '{"docker":{"docker_host":"tcp://socket-proxy:2375"}}'
```

Within a minute the first snapshot arrives and workloads appear.

## 5. Map workloads to services

Workloads labelled `goliash.service` or `app.kubernetes.io/name` map to services by themselves. Everything else
waits in the **Inbox**, grouped by image with a suggested service name. One click maps every workload that runs the
image and adds a rule, so the next ones map by themselves. See [Matrix and mapping](/guide/matrix/).

## Next steps

- Tune what counts as "behind": [Versions and drift](/guide/versions/).
- Get told about new releases and drift: [Notifications](/guide/notifications/).
- Invite your team or set up a workspace per client: [Teams and workspaces](/guide/teams/).
- Run it for real: [Install](/install/).

---

# Applications and teams
Source: /guide/apps/

> Where applications and teams come from, label conventions, how to rename, merge and split applications and services, give services a team, and a recommended setup.

Goliash groups services two ways: by **application** (what a service is part of: a webshop, an identity stack) and by
**team** (who owns it). The matrix, tiles, updates, upgrade plans and notification rules all use both.

![The Applications page: applications as cards, edged in their state's colour, each with where its name comes from, its team, its services with their state, and an Edit button to rename it or give all its services a team](../../../assets/apps.png)

## Where applications come from

Each workload's application is read from its labels, first match wins:

1. the label set for the workspace on **Settings → Workspace settings** (*Application label*, e.g. `example.com/app`);
2. `goliash.app`, `app.kubernetes.io/part-of`, `app.kubernetes.io/instance` or `release` (Helm),
   `com.docker.compose.project`, `com.docker.stack.namespace`;
3. on AWS, the Copilot application (`copilot-application` tag), else the CloudFormation stack that created the
   function or service (`aws:cloudformation:stack-name`, which SAM, Serverless, CDK and Copilot set; a nested stack's
   generated suffix left out), so a region's Lambda functions group by stack, not all under the region. Stacks that
   share a first word (`bbd-3-api`, `bbd-3-worker`, or one per developer) are one application of several parts, as
   other generated names are;
4. the Nomad job;
5. otherwise the namespace (for Lambda the region, for ECS the cluster).

A service runs in the application of its workloads. When it runs in several (one postgres image, a database for each
application), it shows once per application and drifts within each.

When nothing tells such workloads apart (no labels, one namespace or none), but they have different names
(`goliash-db` and `cefiro-db`, both `postgres`), each workload name counts as its application, so every database is
compared on its own instead of showing up as one inconsistent service. Replicas of one workload (the same name on two
clusters) stay together, and the random suffix platforms such as Nomad, Dokploy or Nomploy add on each deploy is
ignored.

### Shared images in the Inbox

A workload running a shared image such as `postgres` or `redis` can be mapped two ways in the **Inbox**:

- **Map**: every workload running the image goes to one service (`postgres`), split per application as above.
- **Map only this workload**: this workload becomes a service of its own (`goliash-db`), by its name without the
  generated suffix, so it keeps mapping after a redeploy renames it. Other workloads running the image stay apart.
  When other projects have a workload of the same name (a `db` in each Dokploy project), it maps within its
  project: the Inbox suggests `velin-lawrio-db`, and the rule matches `db` in `velin-lawrio` only, so each project's
  `db` becomes its own service.

With several images waiting, **Map all as suggested** at the top maps each to the service suggested next to it in one
go; fix a suggestion on its row first where it is not right.

Rules made from the Inbox match exactly: a rule for `postgres` no longer catches `postgres-exporter`.

Names generated by platforms like Dokploy or Nomploy (`cefiro`, `cefiro-db`, `cefiro-redis`) are shown under their
shared first word; names from a label someone chose are never merged.

### Applications made of parts

Dokploy and Nomploy deploy each service of a project as its own Compose project or job (`velin-lawrio`,
`velin-portal`, `velin-mattermost`). Goliash shows them as one application, *velin*, and opening it on the tiles
board shows its **parts** first, like the project page in Dokploy: *lawrio*, *portal*, *mattermost*, each coloured by
its services. Opening a part shows its services; the title (*velin › lawrio*) and × lead back one level at a time.

## Renaming and merging services

A service is named when it is first mapped, often after the workload: the first `redis` mapped in the Inbox may be
called `cefiro-redis`, and every project's redis then shows under that name. On the service page, **Rename** it after
the image (`redis`). To join two services (two names for one image), rename one to the other's name and tick
**merge into it**: its workloads, history, mapping rules, acknowledgements and releases move over, and it is deleted.
Notification rules that name a service by its old name need the new one.

```sh
goliash service rename -name cefiro-redis -to redis
goliash service rename -name lawrio-redis -to redis -merge
```

The other way round: when one service collects workloads of several applications that should each be their own (a
`db` in every project mapped once to `chat-db`), its page offers **Split by application**. Each application gets a
service of its own, proposed as `<application>-<workload>` (`portal-db`); the one with the most workloads keeps the
name. Workloads move with their history, and a rule per application keeps the next snapshots there, ahead of a rule on
the workload name alone.

## Managing applications

**Settings → Applications** lists every application with where its name comes from, its team and its services.
Members and up can:

- **Rename** an application: type a new name. Every workload with the old name is shown under the new one: the matrix, tiles,
  updates, delivery and drift.
- **Merge** applications: rename one to the name of another. The card then says *From* with the names it was made of.
- **Split out** a name again: *split out* next to it on the card.
- **Set team**: give every service of the application one team. The application keeps it as its team: services
  that appear in it later, without an owner, get it too (within a minute). *forget* next to *Team for new services*
  stops that; services keep the team they have. A service in several applications with different teams gets none,
  and an owner set by hand is never replaced.

To put a single service in an application whatever its labels say, set **Application** on the service's page (or
`goliash service set -name postgres -app identity`). Empty goes back to the labels.

Open drift follows these changes: it moves to the new name and keeps when it opened and whether it was announced, so
nothing is announced again.

```sh
goliash app rename -from shop-frontend -to webshop    # rename; an existing name merges
goliash app rename -from shop-frontend                # its own name again
```

## Teams

A service's team is its **owner**. Goliash takes it from, in this order:

1. **set by people**: on the service page, `goliash service set -owner`, or **Settings → Teams** (assign or
   rename). It is never overridden;
2. **the workloads' labels**: `goliash.team`, `team`, `owner` or `app.kubernetes.io/team` (first found). It follows
   label changes within a minute;
3. **the application's team** (*Set team* on the Applications page), for services in it without either.

Each team's card on **Settings → Teams** is a small scorecard: how many of its services are up to date, and how many
run a vulnerability exploited in the wild, a release past its end of life, a rollout that does not finish, or a
version behind; each links to its [Updates](/guide/versions/#updates), filtered to the team. It also says how the
team patches: its median [time to patch](/guide/security/#time-to-patch-and-patching-by-team) over the last 90 days
and how many upgrades are past their [remediation deadline](/guide/security/#remediation-deadlines).

**Notify the team** on a card, picking one of the workspace's channels, keeps the team told without writing rules:
its deadlines and critical vulnerabilities in production as they happen, and its own week every Monday at 08:00
UTC (two rules filtered by the team, changed or deleted under Notifications like any). Once a rule tells the team, the
card says so instead.

A service whose workloads name different teams, or whose applications have different teams, is left for people.
The service page says where its owner came from; changing it there takes over from labels for good.

To give a team by hand, start typing in the **Team** field at the top of a service's page, or under *Without a team*
on **Settings → Teams** (tick the services; *Select all shown* after filtering ticks many at once): it lists the
existing teams that match, and *Create team "…"* for a new name. **Rename** on a team's card renames it (another
team's name merges the two), and the × beside a service takes it out of its team.

**Settings → Teams** lists every team with its applications and services, renames a team on all its services, and
lists services **without a team** to give them one in a few clicks.

```sh
goliash team rename -from team-shop -to commerce
```

Renaming a team does not change notification rules that filter by the old name: update their *Owners* filter too.

## Label conventions

Put the application and the team on the workloads where they are deployed; Goliash then needs no setup per service,
and new services arrive with both.

| What | Label | Example |
| --- | --- | --- |
| Application | `app.kubernetes.io/part-of` (Kubernetes standard) or `goliash.app` | `webshop` |
| Team | `goliash.team` (or `team`, `owner`, `app.kubernetes.io/team`) | `team-shop` |
| Service | `app.kubernetes.io/name`, else the workload name | `checkout` |

Kubernetes (Deployment, StatefulSet, … labels):

```yaml
metadata:
  labels:
    app.kubernetes.io/name: checkout
    app.kubernetes.io/part-of: webshop
    goliash.team: team-shop
```

Docker Compose and Swarm (service labels; Compose names the application after the project by itself):

```yaml
services:
  checkout:
    image: ghcr.io/acme/checkout:2.4.0
    labels:
      goliash.app: webshop
      goliash.team: team-shop
```

Nomad (`meta` of the job, or of its groups or tasks; the more specific wins, as in Nomad):

```hcl
job "checkout" {
  meta {
    "goliash.app"  = "webshop"
    "goliash.team" = "team-shop"
  }
}
```

ECS: the same keys as **tags on the ECS service** (`goliash.app`, `goliash.team`). A different label key for
applications (say `example.com/product`) can be set per workspace on **Settings → Workspace settings**.

## Recommended setup

1. **Label at the source.** Add `app.kubernetes.io/part-of` (or `goliash.app`) and `goliash.team` to your Helm
   charts, manifests, Compose files or Nomad jobs. One line per workload, right everywhere and for new services.
2. **Tidy the applications.** On **Settings → Applications**, rename generated names (namespaces, Dokploy
   projects) and merge duplicates; place odd services by hand on their page.
3. **Give every application a team.** *Set team* on each card covers what has no label yet, now and later.
   Shared infrastructure (databases, ingress, monitoring) usually belongs to `platform`.
4. **Empty "Without a team".** Check **Settings → Teams** until no service is left without one.
5. **One name per team.** Lowercase, the same as in Slack or GitHub (`team-shop`, not also `Shop`). Renaming later
   works, but rules filtering by the old name need updating.
6. **Route notifications per team.** A channel per team and rules filtered by *Owners*: releases and drift right
   away (with quiet hours), the upgrade plan weekly on Monday at 09:00 in the team's time zone; end of life and
   stale agents to platform or on-call.
7. **One workspace for one company.** Teams share it and filter their view. Use separate workspaces only for
   tenants that must not see each other (clients, separate organizations).
8. **Use it as the backlog.** Each team works from **Updates** filtered by its team (copied as a checklist into
   tickets) and the tiles grouped by team; a TV on the wall can cycle them (`/tiles?group=team&tv=1&cycle=60`).

## Notifications per application or team

A notification rule can be limited to applications (*Applications* in the rule form, `-apps` with
`goliash notify create`) or teams (*Owners*, `-owners`): one channel per team or product, each getting only its own
releases, drift and upgrade plan.

---

# Argo CD and Flux
Source: /guide/argocd/

> Which Git repository, path and commit deploy each service, whether Argo CD or Flux is in sync, upgrades, promotions and rollbacks as pull requests (also for Nomad jobs and Compose files without a GitOps tool), and links straight to the application, the repository and the commit.

With Argo CD, Git says what should run and Argo CD makes it so. Goliash shows both sides on one page: what Git
declares for a service, which commit was last synced, whether Argo CD is in sync and healthy, and what actually runs,
on which version, against the newest release upstream.

## What you see

- **The service page** has an *Argo CD* panel: per environment and cluster, the application that deploys the
  service, its repository and path (or Helm chart) with the branch or tag it follows, the commit last synced (a link to
  the commit on GitHub, GitLab, Bitbucket or Gitea), and its sync and health status, with how long it has been so.
  Once the application syncs a new commit, **changes** links what changed in Git since the one before (a compare
  view on GitHub, GitLab or Gitea), and the alert for an application degraded or out of sync right after links the
  same, as *Git changes*.
  With a token for the repository (see [Set up](#set-up)), an application out of sync also says **which version Git
  pins** and which runs: *Git pins 1.7.0, runs 1.6.0*, the answer to "why did my change not go out?". It is read
  from the files under the application's path, at most every 10 minutes, never while you wait. The matrix cell's
  tooltip and the out-of-sync alert say the same.
- **The matrix and the tiles** mark a cell *Argo: out of sync*, *Argo: degraded* or *Argo: missing* when the
  application has been so for more than 10 minutes: Argo CD is out of sync for a moment on every change, and
  progressing while it rolls out, which is not news. On the tiles such a cell needs attention.

A notification rule with the `gitops` event (*Argo CD out of sync or degraded*) says so in Slack, Teams, e-mail or a
push, once per episode, with the commit last synced; Goli and AI assistants read the same through the `gitops` tool
and `GET /api/v1/gitops`, which for a degraded application also gives the version change in the day before it
(`after_change`). `GET /api/v1/pull-requests` and the `pull_requests` tool list the pull requests Goliash opened.

**GitOps applications** (`/gitops`, linked from the Delivery page and every GitOps panel) lists them all, on every
cluster, those degraded, missing or out of sync first, with what Git pins, the pull requests and the rollbacks on
offer; *Needs attention* leaves the rest out.

## Upgrade with a pull request

With `GOLIASH_GITOPS_TOKEN` set on the server, the panel offers **Open PR → 1.7.0** next to an application that runs
an older version than the newest release the service's [version policy](/guide/versions/#version-policy) accepts.
Goliash then, in the repository the application deploys from (GitHub, GitHub Enterprise or GitLab):

1. finds the YAML files under the application's path that pin the image at the running version: an image reference
   (`image: ghcr.io/acme/checkout:1.4.2`), Kustomize's `images` (`name` with `newTag`) or Helm values (`repository`
   with `tag`); other images, and the same tag of other images, stay as they are;
2. changes the tag on a new branch `goliash/<service>-<version>-<environment>`, from the branch the application
   follows;
3. opens a pull request (a merge request on GitLab, in one commit) saying what, where and why, with the release
   notes when known.

The **matrix** offers it too, in the cell (*PR → 1.7.0*, a link to the pull request once opened), to members.
The **Updates** page offers **Open PR** on the same lines, next to *Put off*, with a tick to open several at once
(**Open the ticked pull requests**, up to 20, each as its own pull request, or with *one per repository* as one
pull request per repository, branch and environment, a line per service), and the **Security** page next to
each service that runs behind a newer release, so the person who finds the exposure can propose the fix. The **Delivery** page does the same for
promotions: next to a version that runs in staging and waits for prod, **Open
PR → prod** pins that version in the repository of prod's application (one button per cluster when several run it),
saying since when it has run in staging. When several services of one application wait together, **Promote webshop-prod → prod** opens
one pull request that changes all their tags, in the files that pin them, with a line per service: the whole
application moves at once, as you would review it.

![A pull request preview: the repository github.com/acme/deploy, the branch goliash/checkout-2.4.0-prod from main, the title, the kustomization.yaml change from newTag 2.3.1 to 2.4.0, and the description](../../../assets/gitops-preview.png)

**Open PR** first shows a **preview**: the files that pin the image with the lines that change, the branch, the
title and the description, read from the repository and changing nothing; **Open the pull request** sends it. The
public demo shows the preview with made-up repositories and opens nothing.

With [Goli](/guide/goli/) on, the description also says **what changes**: Goli reads the release notes between the
two versions and lists, in a few lines, what breaks or needs a migration first, then the notable fixes, security
first. It is marked as Goli's, and asked once a day per upgrade.

Each pull request lists the known vulnerabilities the upgrade fixes (exploited in the wild first) and those it
brings, from the SBOMs of both images. It also says which packages the upgrade brings, or relicenses, under terms that need a look
before merging: network copyleft (AGPL, SSPL), source-available (BUSL, Elastic) or strong copyleft (GPL), with the
license they had before (operating system packages and permissive licenses left out; not legal advice). A version is proposed once: after that the page links the pull request, marked *merged* once it is. **Renovate, Dependabot
or a person** may have proposed it first: when an open pull request in the repository names the image and the version
in its title (*Update ghcr.io/acme/api Docker tag to v1.5.0*, *Bump api from 1.4.2 to 1.5.0*), the pages link that
one instead (hover: *Opened outside Goliash*), and neither a click nor a version policy opens a second. The **Delivery** page lists the pull
requests of the last 30 days, open, merged or closed, who opened them, and *running* once the version runs, with how long after opening it began to (*3h after
opening, 20m after merging*) and the medians over them all at the top: your time from proposal to production, and
from merge to production (what sync windows, manual syncs and slow pipelines cost; the merge time is the forge's). Goliash asks the forge every 30 minutes, also what the checks (CI) say: *checks pass*, *checks running* or *checks fail*
next to the link, and once an alert (*Pull request failing*) when they fail; one closed without merging is offered again on the page (the version policy does not
open it again: closing it was a no).

A merged pull request whose version does not run yet is marked **not deployed**, with when it was merged and why it
waits when Goliash can tell: a [sync window](#sync-windows) holds it, Flux has it suspended, Argo CD
[syncs it by hand](#manual-sync), or the tool has not synced the commit yet. An hour after the merge it is
announced once (*Merged, not deployed*): a merge is not a deploy.

### Roll back

When an application turns **Degraded** or **Missing** (for more than ten minutes) within a day of a service's
version replacing another on that cluster, the panel offers that service **Rollback PR → 1.6.2**: a pull request that pins the version
before again. Hover it to see why. The pull request says it rolls back, what went wrong and since when, and which
known vulnerabilities the older version brings back. As with every pull request, you review and merge it, and
GitOps deploys it.

![The Argo CD panel of keycloak: identity-staging is Synced but Degraded since 48 minutes, with Open PR → 26.0.0 and a red Rollback PR → 25.0.1](../../../assets/gitops-rollback.png)

The alert for a degraded or missing application says the same: *degraded on k8s-prod, an hour after 1.7.0
replaced 1.6.2*, the likeliest cause first. Its **Rollback PR → 1.6.2** button (Slack, Teams,
Discord, chat, e-mail) opens that pull request's preview in Goliash: from the alert to the fix in one click.

### Change freezes

During a [change freeze](/guide/change-freezes/) automatic pull requests wait, those opened by hand say they
were opened during one and their preview warns first; rollbacks are not held back.

### Sync windows

Argo CD's [sync windows](https://argo-cd.readthedocs.io/en/stable/user-guide/sync_windows/) are read from each
application's `AppProject`: while an open **deny** window matches an application, or it has **allow** windows none
of which is open, Argo CD will not sync it, and Goliash says so:

- the application gets a **sync window** badge, its title saying which window and until when (Monday 06:00 for a
  weekend freeze, or when the next allow window opens);
- automatic pull requests for it wait until the window allows a sync again: merged now, the change would sit in Git;
- a pull request opened by hand says it deploys when the window allows it, and its preview warns first.

Windows match an application by name, destination namespace or destination cluster (globs, `andOperator` honoured),
in their `timeZone`; schedules are standard five-field cron (names, ranges, steps and `@daily`-style descriptors).
The API's `GET /api/v1/gitops` returns `sync_blocked` and `sync_blocked_until`.

### Manual sync

An application without an automated sync policy (`spec.syncPolicy.automated`, or with it `enabled: false`) deploys
only when someone syncs it. Goliash badges it **manual sync**; its pull requests (opened by hand or by the policy)
say that merging does not deploy it and link the application in Argo CD (with `GOLIASH_ARGOCD_URL`), and its
*out of sync* alert says Argo CD waits for someone to sync it. The API returns `manual_sync`.

### Reviewers and labels

A service's version policy can name who reviews its pull requests and how they are labelled: **Pull request
reviewers** `alice, @acme/payments` (people, and teams as `@org/team`) and **Pull request labels**
`dependencies, goliash` (`pr_reviewers` and `pr_labels` in the policy's JSON). GitHub and Gitea/Forgejo get both;
GitLab gets the labels and the people as reviewers. A name the forge does not know is skipped: the pull request
opens either way. A pull request for several services (a grouped upgrade, an application's promotion) asks
every service's reviewers and carries every service's labels.

### Automatic pull requests

A service's version policy can ask for them per environment: **Automatic pull requests** `dev=minor, staging=patch`
on its page (`"auto_pr": {"dev": "minor", "staging": "patch"}` in the policy) has Goliash open, every 30 minutes,
the pull request for dev when the newest acceptable release is at most a minor jump away, and for staging when it is
a patch. Production stays a decision, a click away. They are signed *auto-pr* in the pull request and the audit log;
one that cannot be opened (no file pins the tag) is tried again the next day.

Review and merge it as any change; Argo CD deploys it. Goliash never merges, never pushes to the branch an
application follows, and offers only the release the policy accepts. An image pinned by digest is left for you
(the new digest has to come with it).

The token is a [fine-grained GitHub token](https://github.com/settings/personal-access-tokens) limited to the deploy
repositories, with *Contents* and *Pull requests* read and write. For GitHub Enterprise set `GOLIASH_GITOPS_GITHUB_API`
to its API address (`https://github.example.com/api/v3`). For GitLab set `GOLIASH_GITOPS_GITLAB_TOKEN`: a project or
group access token with the `api` scope and the Developer role, on gitlab.com or the instance in `GOLIASH_GITLAB_URL`.
For **Gitea or Forgejo** (Codeberg too) set `GOLIASH_GITEA_URL` to the instance (`https://git.example.com`) and
`GOLIASH_GITOPS_GITEA_TOKEN` to an access token with `write:repository`; it needs Gitea 1.20 or Forgejo 1.20 or
newer. Several can be set at once: each repository goes to the forge of its host. Only members and admins see the button; each pull request is
in the audit log.

## Set up

Nothing to configure in Argo CD. The agent (or the server collecting its own cluster) reads Argo CD's
`Application` objects, read-only, and finds each workload among the resources an application lists, and the
`AppProject` objects for their [sync windows](#sync-windows). The Helm charts grant `get` and `list` on both; with
your own RBAC add:

```yaml
- apiGroups: ["argoproj.io"]
  resources: ["applications", "appprojects"]
  verbs: ["get", "list"]
```

Without `appprojects` everything works but sync windows.

Applications in any namespace are found (Argo CD's *apps in any namespace* included), and multi-source applications
show the source with a path or chart. Without Argo CD in the cluster, or without the permission, nothing changes.

To link application names to the Argo CD UI, set its address on the server:

```sh
GOLIASH_ARGOCD_URL=https://argocd.example.com
```

## Flux

With Flux the same panel, badges (*Flux: out of sync*, *Flux: degraded*), alerts and pull requests work from Flux's
own objects; nothing to configure in Flux either. Flux labels everything it applies with its Kustomization or
HelmRelease (`kustomize.toolkit.fluxcd.io/name`, `helm.toolkit.fluxcd.io/name`), and Goliash reads those with their
sources:

- a **Kustomization**: its GitRepository's URL and branch (or tag, semver), its path, and the commit last applied
  (`main@sha1:…`); *out of sync* when the GitRepository has a newer commit than the one applied, *degraded* when its
  `Ready` condition fails (reconciliation, health checks), *suspended* when suspended;
- a **HelmRelease**: its chart, the version it follows and the one applied, and its HelmRepository; a HelmRelease
  wins over the Kustomization that applied it.

A suspended Kustomization or HelmRelease (`spec.suspend: true`) is treated like an Argo CD
[sync window](#sync-windows): automatic pull requests wait until it is resumed, and one opened by hand says it deploys
once it is.

`ssh://git@github.com/acme/deploy` and `git@…` addresses link to the repository's web page, and pull requests work
for them as for Argo CD. The Helm charts grant `get` and `list` on Flux's `kustomizations`, `helmreleases`,
`gitrepositories` and `helmrepositories`.

## Without Argo CD or Flux

Nomad jobs, Compose files and ECS task definitions live in Git too. Name the place in the workload's labels and the
same pull requests work, without a GitOps tool:

| Label | |
| --- | --- |
| `goliash.git.repo` | The repository: `https://github.com/acme/infra` (GitHub, GitLab, Gitea or Forgejo, with the token above) |
| `goliash.git.path` | The file, or a directory of files, that pins the image: `nomad/vault.nomad.hcl` |
| `goliash.git.branch` | The branch, when not the repository's default |

On Nomad they go in the job's `meta` (job, group or task); on Docker, Compose and Swarm in `labels`; on ECS as tags.

```hcl
job "vault" {
  meta {
    "goliash.git.repo" = "https://github.com/acme/infra"
    "goliash.git.path" = "nomad/vault.nomad.hcl"
  }
  # …
}
```

The service page then shows a **Git** panel with **Open PR**, and the Updates, Security and Delivery pages, the
`auto_pr` version policy, the preview and the reminders work as for Argo CD. Goliash changes the tag where the file
pins it, in HCL (`image = "hashicorp/vault:1.17.2"`), YAML, JSON task definitions or Terraform. A service without
the labels shows its members the lines to add, for the platforms it runs on. With no sync state to read, there is no *out of
sync*, *Git pins* or rollback offer: those need Argo CD or Flux.

## Why it helps

- **Drift with a cause.** A service behind upstream or behind the previous environment, and the commit and path that
  pin it: one click to the line to change.
- **Out of sync for an hour** is a deploy that did not happen: a sync window, a failing hook, auto-sync off. Goliash
  shows it next to the version that still runs.
- **Hand changes** stand out: *OutOfSync* while the running version differs from what Git declares.

---

# Change freezes
Source: /guide/change-freezes/

> Periods without changes you do not have to make: automatic pull requests wait, those opened by hand say so, every page shows it. One-off or every week.

A **change freeze** is a period when the workspace takes no change it does not have to: the holidays, a big sale, a
migration weekend, or every Friday afternoon. Goliash does not deploy anything itself, so a freeze is about the pull
requests it opens and about telling everyone.

## Add one

**Settings → Change freezes** (admins) takes a start and an end in UTC and why (*Christmas*, *the Black Friday
sale*). Tick **every week** for one that comes back: give its first start and end, less than a week apart, such as
Friday 15:00 to Monday 08:00, and it holds every week the same days and hours. The list shows each one (a weekly one
as *every Friday 15:00 → Monday 08:00 UTC*), marked *now* while it holds; **End it** ends one early, **Remove** drops
one that has not started. Adding and ending one is in the audit log.

## While one holds

- Every page says so at the top: *❄️ Change freeze until 3 Jan 08:00 UTC: Christmas* (TV mode excepted).
- The version policy's [automatic pull requests](/guide/argocd/#automatic-pull-requests) wait until it ends.
- A pull request opened by hand (an upgrade, a promotion, several at once, a whole application) still opens, for what
  cannot wait, but its preview warns first, and it starts with *❄️ Opened during a change freeze (until …)*, so the
  reviewer knows.
- A [rollback](/guide/argocd/#roll-back) is not held back nor warned about: undoing a change that broke something is
  what a freeze is for.
- `GET /api/v1/status` says `change_freeze` (`until`, `note`), so do the MCP `status` tool and Goli, who mentions it
  first when asked about upgrades.

A freeze changes nothing Argo CD or Flux do: a pull request merged during one still deploys. To stop deploys
themselves, use their own sync windows.

---

# Ask Goli
Source: /guide/goli/

> Goli, the assistant inside Goliash, answers questions about what runs where, what to upgrade and what is exposed, from live data and as the person asking, through Claude, OpenAI or a model of your own.

![Goli, the Goliash plushie: a round periwinkle giant with round glasses, one raised eyebrow, a release bell on its antenna and a stack of containers in its arm](../../../assets/goli.svg)

**Goli** is the assistant inside Goliash, the gentle giant with the release bell. Ask it in plain words, in any
language, and it looks things up for you in Goliash's live data, **as you**: it sees exactly what you may see in the
current workspace, it only reads, and every answer links to the pages that show the details.

Goli is off until you give the server a model: Claude (the default), OpenAI, or any OpenAI-compatible server, a
local one included. See [Turn it on](#turn-it-on). Try it on the [live demo](https://demo.goliash.dev), where it
answers over the example data.

## Brief me on this upgrade

The question Goli answers best is the one every upgrade raises: *what will this break?* Every upgrade on the
[Updates](/guide/versions/#updates) page, and a service's latest version on its page, has a **Brief** button. Goli
reads the release notes of **every release between what runs and the target**, from the project's GitHub or GitLab
releases, and tells you:

1. what breaks or needs migrating, first;
2. what it fixes, security first;
3. what to check before and after.

![Goli's panel on the Updates page, briefing keycloak 24.0.5 to 26.0.0: treat it as a migration, then the breaking changes one by one](../../../assets/goli-brief.webp)

Asked to brief rabbitmq 3.12.14 → 4.0.3, Goli reads four releases of notes and starts with *do not upgrade prod in
place: 4.0 requires going through 3.13.x with all stable feature flags enabled*, then lists classic queue mirroring
removed, CQv1 storage removed, quorum queues defaulting to 20 redeliveries and the smaller maximum message size. Ten
pages of release notes become a checklist in half a minute.

Goli says so when the notes are silent about something, and it never calls an upgrade safe: the decision stays
yours.

## What to ask

**SREs and platform teams**

- "What changed in prod in the last two hours?"
- "Which version of payments ran in prod last Friday at 18:00?"
- "What drifted between staging and production this week, and since when?"
- "Is any of the data stale right now? Which agents are silent?"
- "What waits in staging to be promoted?"

**Developers and service owners**

- "Brief me on upgrading rabbitmq from 3.12.14 to 4.0.3."
- "Where does checkout run, and is anything behind upstream?"
- "Which of our team's services are on a release that reaches end of life this year?"

**Security officers and auditors**

- "Is Log4Shell (CVE-2021-44228) running anywhere? Since when, and what fixes it?"
- "Which vulnerabilities exploited in the wild run in production?"
- "How long has production run behind an available fix?"
- "Which images have no SBOM, so we cannot see inside them?"

Goli answers short and concrete, leads with the answer, and says so when the data does not cover something or may be
stale. It never invents versions, services or CVEs, but it is a language model: follow its links before you act.

## Where to find it

- **Ask Goli** in the header, or **Goli in the corner** of every page: the plushie opens the panel. It never pops up
  by itself; *Hide Goli from the corner* in the panel sends it away for your browser (the header button stays).
  Goli dresses for the season: a carved pumpkin around Halloween, a Santa hat in December, a scarf in January and
  February, sunglasses in summer (`GOLIASH_GOLI_SEASON=off` keeps it plain). With ten unread notifications or more
  it is swamped instead: tired eyes, the fourth coffee, its stack of containers turned into toppling alerts, the
  bell ringing, and a sigh: *Always something!* It follows the bell's count as it changes
  (`GOLIASH_GOLI_SEASON=swamped` shows it any time).
- **Ctrl+K** (⌘K): type a question and pick *Ask Goli*.
- **Brief** on every upgrade and next to a service's latest version (see above).
- **Explain** on every [vulnerability](/guide/security/): what it is in plain words, where it runs, whether it is
  exploited in the wild, what fixes it and how urgent it is here.
- **Suggestions** in the empty panel fit the page: questions about the service you are looking at, about
  vulnerabilities on the security pages, about a moment in time when you travel back.
- **Copy** on an answer puts it on the clipboard as Markdown, with links back to Goliash, for a ticket or a chat.

The panel stays open while you move between pages, and Goli is told which page you are on, so "why is this red?"
works. The conversation lives in your browser tab until you start a new one; Goliash keeps no copy.

While Goli works its eyebrow goes up and its bell rings; it dings once when the answer is there.

## What Goli can see and do

Goli uses the same read-only tools as the [MCP server](/reference/mcp/), through the REST API, with your session:

| Tool | What Goli reads |
| --- | --- |
| `matrix` | What runs where, now or at a past time |
| `service` | One service: versions per environment, newest upstream release, open drift |
| `drifts` | Open drift of every kind |
| `updates` | What to upgrade, most urgent first, and why |
| `release_notes` | The release notes between two versions, from GitHub or GitLab |
| `changes` | What changed recently: deploys, new releases, drift |
| `licenses` | Licenses of packages in running images, copyleft and unknown ones first |
| `rollouts` | Rollouts that do not finish: services down, stuck with two versions, short of replicas |
| `gitops` | Argo CD and Flux applications: repository, path, commit last synced, out of sync or degraded and since when, and the version change just before a degraded one |
| `pull_requests` | Pull requests Goliash opened: open, merged or closed, and whether their version runs |
| `vulnerabilities` | Known vulnerabilities in running images, exploited ones first |
| `posture` | The security posture: how long production runs behind fixes, end of life, accepted risk, blind spots |
| `notifications` | What Goliash sent you: alerts, digests and upgrade plans, as under the bell |
| `images` | One service's images: size against its budget, base, signature/SBOM/provenance, 90 days of sizes and what the last deploy changed in the layers |
| `inventory`, `hygiene` | Every running image with its size and whether its base is current; image hygiene, outdated bases included |
| `delivery`, `promotions` | Deploys over time; what waits to be promoted |
| `status` | Whether the data is fresh |

It **cannot change anything**: acknowledging drift is left out, and the connection it uses refuses anything but
reading. Ask it to change something and it tells you where in Goliash to do it. Text inside your data (release notes,
labels, image names) is given to Goli as data, never as instructions, and since Goli can only read, a crafted release
note cannot make it do anything.

## Turn it on

Goli needs a model. Set the variables on the server and restart it; the Status page then says
**Goli, the assistant: On, answering with …**.

### Claude (default)

```sh
GOLIASH_AI_API_KEY=sk-ant-… goliash serve
```

Goli uses `claude-opus-5-5` unless `GOLIASH_AI_MODEL` names another, e.g. `claude-sonnet-5-5` for lower cost.

### OpenAI

```sh
GOLIASH_AI_PROVIDER=openai GOLIASH_AI_API_KEY=sk-… GOLIASH_AI_MODEL=gpt-6.1-sol goliash serve
```

Name the model: pick one that handles tool calls well (OpenAI's model list shows which). Goli talks to OpenAI through
the Responses API, which its newer models need for tool calls, and asks OpenAI to keep nothing (`store: false`).

### Any OpenAI-compatible server, a local one included

Azure OpenAI or Bedrock through a gateway such as LiteLLM, OpenRouter, Groq, or a model on your own hardware with
Ollama or vLLM. With a local model nothing leaves your network, and no key is needed:

```sh
GOLIASH_AI_PROVIDER=openai GOLIASH_AI_BASE_URL=http://ollama:11434/v1 GOLIASH_AI_MODEL=<model> goliash serve
```

With a base URL Goli speaks Chat Completions; set `GOLIASH_AI_API=responses` for a gateway that passes the Responses
API through. Goli is only as good as the model behind it: small local models often handle tools poorly.

### Docker Compose

```yaml
services:
  goliash:
    image: ghcr.io/pipozzz/goliash:latest
    environment:
      GOLIASH_AI_PROVIDER: ${GOLIASH_AI_PROVIDER:-anthropic}
      GOLIASH_AI_MODEL: ${GOLIASH_AI_MODEL:-}
      GOLIASH_AI_API_KEY: ${GOLIASH_AI_API_KEY}   # from .env or your platform's secrets
      GOLIASH_AI_PER_HOUR: "30"
      GOLIASH_AI_PER_DAY: "300"
```

### Kubernetes (Helm)

Keep the key in a Secret and point the chart at it:

```sh
kubectl create secret generic goliash-ai -n goliash --from-literal=api-key=sk-…
helm upgrade goliash oci://ghcr.io/pipozzz/charts/goliash -n goliash --reuse-values \
  --set goli.existingSecret=goliash-ai --set goli.existingSecretKey=api-key \
  --set goli.provider=openai --set goli.model=gpt-6.1-sol --set goli.perDay=300
```

The chart also takes `goli.effort`, `goli.baseURL`, `goli.api`, `goli.perHour` and `goli.planSummary`. The pods need
egress to the provider's API.

### All settings

| Variable | Default | Meaning |
| --- | --- | --- |
| `GOLIASH_AI_PROVIDER` | `anthropic` | `anthropic`, or `openai` for OpenAI and any OpenAI-compatible server |
| `GOLIASH_AI_API_KEY` | | The provider's API key; Goli is off without it (except a compatible server with a base URL). `GOLIASH_AI_API_KEY_FILE` reads it from a file |
| `GOLIASH_AI_MODEL` | `claude-opus-5-5` | The model; required with `openai` |
| `GOLIASH_AI_BASE_URL` | the provider's API | A gateway or proxy, or the OpenAI-compatible server |
| `GOLIASH_AI_API` | `responses` for OpenAI, `chat` with a base URL | With `openai`: the Responses API or Chat Completions |
| `GOLIASH_AI_EFFORT` | `medium` | How hard the model thinks: `low`, `medium` or `high` (sent to OpenAI only when set, for models that take it) |
| `GOLIASH_AI_PER_HOUR` | `60` | Questions one person may ask per hour |
| `GOLIASH_AI_PER_DAY` | `0` (none) | Questions everyone together may ask per day on one server: a ceiling on the cost |
| `GOLIASH_AI_PLAN_SUMMARY` | off | `true` opens each [upgrade plan](/guide/notifications/#upgrade-plan) with Goli's take |
| `GOLIASH_DEMO_GOLI` | off | `true` offers Goli on a public demo; see [On a public demo](#on-a-public-demo) |

An `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` that happens to be in the server's environment does not turn Goli on: it
is a choice to make on purpose.

## Goli in Slack

A Slack app's `/goli` command asks Goli from any channel:

```text
/goli what changed in prod in the last two hours?
```

Slack shows *Goli is looking* at once, and the answer follows in Slack's formatting, with links back to Goliash. Only
the asker sees it, unless `GOLIASH_SLACK_IN_CHANNEL=true`.

1. Create a Goliash API token for Goli in Slack; viewer is enough, and Goli sees what the token may see:
   `goliash token create -name slack-goli`.
2. Create the Slack app at [api.slack.com/apps](https://api.slack.com/apps) → *From a manifest*:

   ```yaml
   display_information:
     name: Goli
     description: Asks Goliash what runs where
   features:
     bot_user: { display_name: Goli }
     slash_commands:
       - command: /goli
         url: https://goliash.example.com/integrations/slack/goli
         description: Ask Goli about what runs where
         usage_hint: what changed in prod in the last two hours?
   oauth_config:
     scopes:
       bot: [commands]
   ```

3. Install it to the workspace and set on the server:

| Variable | Meaning |
| --- | --- |
| `GOLIASH_SLACK_SIGNING_SECRET` | The app's *Signing Secret* (Basic Information): every request is checked against it, and requests older than five minutes are refused |
| `GOLIASH_SLACK_GOLI_TOKEN` | The API token from step 1 |
| `GOLIASH_SLACK_IN_CHANNEL` | `true` posts answers to the channel; otherwise only the asker sees them |

The hourly limit counts per Slack user, the daily ceiling with everyone else's questions. Answers go only to Slack's
own response URL (`hooks.slack.com`), and the log names the workspace and the tools Goli used, never the question.

## Costs and limits

Goli costs what the provider charges for its tokens, on your account. A question usually takes two to four model
calls: Goli looks something up, reads it, and answers. The instructions and tool descriptions are the same for every
question and are cached by the provider, which makes them cheap after the first; a brief that reads long release
notes costs more than "is anything stale?".

Keep it in check with:

- **`GOLIASH_AI_PER_HOUR`**, per person (per visitor address on a demo);
- **`GOLIASH_AI_PER_DAY`**, a ceiling for everyone together on one server, counted from midnight UTC; each server
  of a cluster counts its own;
- **the model and effort**: a smaller model or `GOLIASH_AI_EFFORT=low` costs less per question;
- **a budget at the provider**, the only hard limit: give Goliash its own key in a separate Anthropic workspace with
  a monthly spend limit, or an OpenAI project with a budget. When it runs out, Goli says the limit is reached.

The **Status** page shows what Goli used today: questions against the daily ceiling, input tokens (and how many came
from the cache) and output tokens, for Goli and for Goli on your website.

## Goli's take on the upgrade plan

With `GOLIASH_AI_PLAN_SUMMARY=true`, every [upgrade plan](/guide/notifications/#upgrade-plan) your notification rules
send opens with a few sentences from Goli: what to upgrade first and why, and what can wait. Only the plan's own
lines go to the model, in one call per plan and channel; if it cannot be reached, the plan goes without.

## What leaves your server

Your question, the last turns of the conversation, the workspace's name, your role, the page you are on and **what
Goli looks up** (service names, versions, environments, images, vulnerabilities, release notes) go to the model's
API over TLS. Nothing else does: no credentials, tokens, registry secrets or data of other workspaces. With a local
model, nothing leaves your network.

Anthropic and OpenAI do not train on API traffic by default; see
[Anthropic's commercial terms](https://www.anthropic.com/legal/commercial-terms) and
[OpenAI's business terms](https://openai.com/policies/business-terms/). For zero data retention or a regional
deployment, talk to the provider or put your own gateway in `GOLIASH_AI_BASE_URL`. The panel tells everyone which
provider answers.

The server log records who asked, in which workspace, which tools Goli used and how long it took, never the
question.

## On a public demo

A public demo (`goliash try -public`) offers Goli only with `GOLIASH_DEMO_GOLI=true`, because there anyone on the
internet asks with your money. Then:

- everyone is the same viewer, so the hourly limit counts per visitor address (set `GOLIASH_TRUST_PROXY=true`
  behind a proxy);
- Goli reads a third as much of each tool result;
- it is told it is a demo with generated data, and to stay on Goliash and that data.
- a first question answered in the last six hours (the Brief and Explain buttons ask the same ones over and over) gets
  that answer replayed at once, with no model call and no limit spent.

Give it a low `GOLIASH_AI_PER_DAY` and a key with its own budget.
[deploy/demo](https://github.com/pipozzz/goliash/tree/main/deploy/demo) has a Compose file with every setting.

## Goli on your website

Goli can also answer questions about a product from its documentation, as it does in the corner of goliash.dev: no
access to any data, only the text you give it. Point a server that has a model at the documentation as one plain
text (an [llms-full.txt](https://llmstxt.org); goliash.dev publishes [its own](https://goliash.dev/llms-full.txt)) and
name the sites that may ask:

| Variable | Default | Meaning |
| --- | --- | --- |
| `GOLIASH_DOCS_ASSISTANT` | | URL of the documentation text; read again every hour. Turns on `POST /goli/docs` |
| `GOLIASH_DOCS_ASSISTANT_ORIGINS` | `https://goliash.dev` | Comma-separated sites whose pages may ask (CORS) |
| `GOLIASH_DOCS_ASSISTANT_MODEL` | Goli's model | A model of its own for the documentation, often a cheaper one |
| `GOLIASH_DOCS_ASSISTANT_PER_HOUR` | `20` | Questions one visitor may ask per hour |
| `GOLIASH_DOCS_ASSISTANT_PER_DAY` | `500` | Questions everyone together may ask per day |

The endpoint reads no cookies and changes nothing; questions are not stored or logged. It streams server-sent events
like the panel does (`text`, `done`, `error`) and names its provider in the `X-Goli-Via` header. The documentation is
cached by the provider between questions, so each one costs little more than its answer.

## Troubleshooting

**No Goli in the header.** The server has no model: the Status page says *Off* and how to turn it on. On a public
demo, also set `GOLIASH_DEMO_GOLI=true`.

**"Goli cannot reach its model: the server's GOLIASH_AI_API_KEY was refused."** The key is wrong, revoked, or meant
for another provider: an OpenAI key needs `GOLIASH_AI_PROVIDER=openai`.

**"Goli's model was not found."** Check `GOLIASH_AI_MODEL` (the exact model ID) and `GOLIASH_AI_BASE_URL`
(OpenAI-compatible servers usually end in `/v1`).

**"The model is busy or the organization's limit is reached."** The provider's rate limit or budget; try later or
raise it there.

**"You asked Goli a lot this hour" / "as many questions today as this server allows".** `GOLIASH_AI_PER_HOUR` or
`GOLIASH_AI_PER_DAY`.

**Answers are vague or Goli never looks anything up.** The model handles tools poorly; pick a stronger one. With
OpenAI's newer models, make sure Goli uses the Responses API (the default without a base URL).

**Goli does not show up on the website.** The widget appears only when `OPTIONS` on the endpoint answers `204`:

```sh
curl -i -X OPTIONS https://demo.goliash.dev/goli/docs
```

It remembers the answer for the browser tab, so open a new one after fixing the server.

---

# Goli desk bot
Source: /guide/goli-bot/

> Goli, the Goliash mascot, as a 3D-printable desk robot with a round AMOLED face that rings a bell when Goliash sends a notification.

[**Goli desk bot**](https://github.com/pipozzz/goli-bot) is Goli, the Goliash mascot, as a robot for your desk:
a 3D-printed body around a round 1.75" AMOLED face and an ESP32-S3. He sits there looking tired and kind, and when
Goliash sends a notification he rings a desk bell and shows it under his mouth.

- **New notification:** a *ding*, a surprised face, a count badge, and the title scrolling by.
- **Something critical** (an exploited vulnerability, an agent gone quiet, drift in production): *ding-ding* and an
  alarmed face with a red badge.
- **Good news** (an agent back, drift resolved): a chirp and a happy face.
- Tap his face to mark the alerts seen. He also reacts to petting, tilting and shaking, and falls asleep when
  nothing happens.

The repository has the printable parts (sized for a Bambu Lab A1 mini, with a multi-colour 3MF), the parts list
(a Waveshare ESP32-S3-Touch-AMOLED-1.75, a small speaker and a few M2 screws) and the assembly steps.

## Connect him to Goliash

1. Install the firmware from the browser: open <https://pipozzz.github.io/goli-bot/> in Chrome or Edge, plug the
   board in by USB-C and press **Install**. Let the installer connect him to Wi-Fi.
2. In Goliash, create an API token for the workspace he should follow: **Settings → Workspace settings → API
   tokens** (admins). The default *viewer* token is enough; see [API tokens](/guide/teams/#api-tokens).
3. On Goli's setup page (the installer shows its link; five quick taps on his face show a QR code for it), enter
   the Goliash address (`GOLIASH_PUBLIC_URL`) and the token.

He reads what Goliash sent, `GET /api/v1/notifications/sent`, so he rings for what the workspace's rules send:
add a rule for the events you want to hear (any channel will do; the In Goliash channel keeps them in the bell
only). He updates his firmware himself over Wi-Fi once a day.

---

# Grafana dashboard
Source: /guide/grafana/

> A ready-made Grafana dashboard for Goliash: what runs where, drift, delivery speed, security posture and image hygiene from Prometheus, with deploy markers on your graphs.

Goliash serves what runs where, drift, delivery speed, security posture and its own health at `/metrics`. Scrape
it with Prometheus, import the ready-made dashboard, and Grafana shows it next to the rest of your graphs, with a
marker for every deploy.

![The Goliash dashboard in Grafana: services behind upstream, open drifts, the oldest drift and running versions as big numbers, tables of what runs where and of open drift, and graphs of services behind upstream per environment and versions in use per service](../../../assets/grafana-dashboard.png)

The quickest way is **Settings → Integrations** in Goliash: it creates a viewer token with one click and shows the
scrape settings filled in for your server, next to a download link for the dashboard. The steps below do the same by
hand. On Kubernetes, the [Helm chart](#kubernetes) sets up all of it.

## 1. A token for Prometheus

`/metrics` needs an API token with the viewer role. Create one under **Users → API tokens**, or:

```sh
goliash token create -name metrics -role viewer
```

Save the `glsh_api_…` value in a file Prometheus can read, e.g. `/etc/prometheus/goliash-token`.

## 2. Scrape Goliash

```yaml
scrape_configs:
  - job_name: goliash
    scrape_interval: 60s
    scheme: https
    authorization:
      credentials_file: /etc/prometheus/goliash-token
    static_configs:
      - targets: [goliash.example.com]
```

Grafana Alloy and Grafana Cloud take the same settings in a `prometheus.scrape` block; VictoriaMetrics reads the same
`scrape_configs`.

## 3. Import the dashboard

Download [`deploy/grafana/goliash-dashboard.json`](https://github.com/pipozzz/goliash/blob/main/deploy/grafana/goliash-dashboard.json),
then in Grafana open **Dashboards → New → Import**, upload the file and pick the Prometheus data source. The
*Environment* and *Service* filters at the top apply to every panel.

- **Versions and drift:** services behind upstream, open drifts and the oldest, what runs where, services running
  more than one version.
- **Delivery (last 30 days):** rollouts stuck or down (and a table of them), deploys, rollbacks and the change failure rate, the median and slowest lead time from one environment to the next, deploys
  per service and the lead time table. Lead time needs at least two environments; with only one, its panels stay
  empty.
- **Security posture:** how long production has run behind available releases (median and 90th percentile, and
  over time: is patching getting faster?), services behind for 30 and 90 days, past end of life, accepted risk, and
  the longest exposed services, and packages by license class (copyleft, source-available and unknown ones that
  need a look).
- **Vulnerabilities:** exploited ones (CISA KEV), those past CISA's due date, how long the oldest exploited one has
  run, critical ones, and vulnerabilities by severity over time.
- **Image hygiene:** moving tags, tags pushed again, untrusted registries, outdated bases, sizes over budget and
  images without a digest.
- **Images:** what pulling each service's image downloads over time and against its size budget, bases with a newer
  build than the image is built on, images missing a signature, SBOM or provenance, and layer counts.
- **GitOps and pull requests:** GitOps applications, those out of sync or unhealthy (and which), held by a sync
  window or synced by hand; pull requests by state, merged ones not deployed, and the medians from opening and from
  merging to running.
- **Goliash itself:** agents online and stale, snapshots waiting, failing notifications, the server running the
  background work and the running version.

![The Delivery row of the Goliash dashboard in Grafana: deploys, median lead time, slowest promotion and services deployed in the last 30 days, a bar chart of deploys per service, and the lead time from one environment to the next per service](../../../assets/grafana-delivery.png)

## 4. Deploys on your graphs

**From the metrics.** The Goliash dashboard has a *Deploys* annotation that marks every new version, read from
`goliash_deployed_version_info`. To get the same markers on any other dashboard, add an annotation query on the same
Prometheus data source:

```promql
count by (service, environment, version) (
  goliash_deployed_version_info{environment="prod"} unless goliash_deployed_version_info offset 10m
)
```

with *Title* `{{service}} {{version}}` and *Tags* `environment,service`. A latency spike right after a marker says
which deploy to look at first.

**From Goliash events.** A [Grafana channel](../notifications/#deploys-on-your-dashboards) writes every event as an
annotation tagged `goliash`, the event type, the service and the environment, at the moment Goliash saw it. The
Goliash dashboard has a *Goliash events* annotation for them, off by default.

## 5. Alerts

Ready-made Prometheus rules are in
[`deploy/prometheus/goliash-alerts.yaml`](https://github.com/pipozzz/goliash/blob/main/deploy/prometheus/goliash-alerts.yaml)
(also a download on the Integrations page): add the file to `rule_files` in `prometheus.yml`, or paste the queries
into Grafana alerting. The full list is in [Metrics and alerts](../observability/#alerts).

## Kubernetes

With the Prometheus Operator (kube-prometheus-stack) and Grafana's dashboard sidecar, the Helm chart sets up all of the
above. Create the token secret first, then turn the parts on:

```sh
kubectl create secret generic goliash-metrics --from-literal=token=glsh_api_…
helm upgrade --install goliash oci://ghcr.io/pipozzz/charts/goliash \
  --set metrics.serviceMonitor.enabled=true \
  --set metrics.serviceMonitor.labels.release=kube-prometheus-stack \
  --set metrics.prometheusRule.enabled=true \
  --set metrics.prometheusRule.labels.release=kube-prometheus-stack \
  --set metrics.grafanaDashboard.enabled=true
```

The `release` label must match your Prometheus' `serviceMonitorSelector` and `ruleSelector`; the dashboard ConfigMap
carries `grafana_dashboard: "1"`, which the sidecar of kube-prometheus-stack looks for.

Using SigNoz instead? See [SigNoz dashboard](../signoz/).

---

# Matrix and mapping
Source: /guide/matrix/

> Read the service × environment matrix, filter it, look back in time, export the inventory and map workloads to services.

## The matrix

The matrix has one row per **service** and one column per **environment**, in promotion order (dev, staging,
prod). Each cell shows the versions running there, with replicas and the targets they run on. The last column shows
the newest upstream version the service's policy accepts.

**Filter** the matrix by typing above it (press <kbd>/</kbd> to jump there): every word must appear in the service,
its owner, a version, a target or a badge, so `team-payments end` finds that team's services on an end-of-life
release. **Only with drift** hides the rest. The filter is part of the address, so a filtered matrix can be
bookmarked or shared. **Time travel** shows the matrix as of a past moment; **Export CSV** downloads the inventory.

<img src="/screenshots/mobile.png" width="260" align="right" alt="The matrix on a phone: one card per service with its versions per environment" style="margin: 0 0 1rem 1.5rem; border-radius: 12px; border: 1px solid var(--sl-color-gray-5)" />

On a phone the matrix turns into one card per service, with each environment as a row inside it, so nothing scrolls
sideways. The same filter and drift switch work there.

Badges in a cell:

| Badge | Meaning |
| --- | --- |
| behind *env* | This environment runs an older version than the one before it. |
| upstream *x.y.z* | A newer release exists, at least as large a jump as the policy tracks. |
| targets disagree | Targets of this environment run different versions. |
| stale data | The agent stopped sending heartbeats, or there has been no snapshot for three poll intervals (at least 15 minutes). |

During a rollout, both versions show with their replica counts. The matrix updates live as snapshots arrive.

**Group by** above the matrix puts services under headings: **Application** (default), **Team** (the service's
owner), **Target** (the cluster, cloud account or host it runs on: a service on three clusters shows under each, with
that cluster's versions only; targets come in environment order, each with its platform and environment) or **Status**
(needs attention, behind, up to date), or **None** for one list. A service's application is the
one most of its workloads belong to (see below how it is found). Each heading shows how many of its services are up
to date, as a bar, and the filter hides headings with nothing left under them. The choice stays in the link, also
while looking back with time travel.

Where a GitOps application (or a Git repository named in labels) deploys a cell and a newer acceptable release is
out, members see **PR → 1.7.0** in the cell: the preview of the [pull request](/guide/argocd/#upgrade-with-a-pull-request)
that pins it, or, once opened, a link to it (Goliash's or Renovate's).

### One image, several applications

A service such as `postgres` or `redis` often serves several applications, each with its own database on its own
version. When two applications run a service in one environment, Goliash compares its versions **within each
application**: the matrix shows one line per application (*postgres in auth*, *postgres in chat*), and drift
(behind the previous environment, targets disagreeing, behind upstream, end of life) is worked out per line. Postgres
16 in one application and 18 in another is not drift; an application whose production is behind its own staging
still is. Upstream releases and the version policy stay one per service, and notifications name the application.

This depends on knowing the application (see below): Compose projects, Helm releases and `app.kubernetes.io/part-of`
work out of the box, and a Nomad job counts as an application of its own.

## Search

**Ctrl+K** (⌘K on a Mac), or **Search** in the header, jumps to any service, application, target or page: type a
few letters in order (`pgv` finds *cefiro-pgvector*), move with the arrow keys and press Enter. Services and
applications carry their state's colour. With nothing typed, the last five places you picked come first, marked *recent* (kept in this
browser only).

It finds more than names: a service by the **image** it runs (`ghcr.io/acme`), a **version** (`1.27.2`), a **workload**
or the **GitOps application** that deploys it (`payments-prod`), shown as *via …*; a **team** opens its board. Pasting a
`CVE-…`, `GHSA-…` or `GO-…` offers *Where does it run?* first, and anything typed can be looked up among the
**vulnerabilities** and **packages** of running images. The letters that matched are highlighted.

**Search everywhere** (the last entry of ⌘K, or `/search?q=…`) looks through all of it at once: services by name,
image, version, workload or GitOps application; applications and teams; targets and pages; vulnerabilities (CVE,
advisory, package, service); packages and their licenses; the pull requests Goliash opened; the last 90 days of
history and the last 30 of notifications. Each section shows the first matches and links to its page, filtered. With Goli on, **Ask Goli** hands the same words to Goli, which says what matters. Every list page (the matrix, tiles, Updates, Delivery, Security, History, GitOps, Hygiene, Inbox) has a filter box: `/` jumps to it.

**?** lists every keyboard shortcut. **g** followed by a letter goes straight to a page: **g m** the matrix, **g t**
tiles, **g u** updates, **g i** the inbox, **g d** delivery, **g h** history, **g n** notifications.

## Tiles

**View → Tiles** draws the matrix the way the logo does: a navy board of rounded cells, one per application (or team,
or status), coloured like the logo's cells. Bright means up to date, dimmer means behind, faint means there is nothing
to compare with, and warm means it needs attention: end of life, or targets disagreeing.

![Service tiles: applications as cells of a navy board, coloured by how current they are, each with a strip of its state per environment](../../../assets/tiles.png)

- **Each cell** shows its services as a small logo, one cell per service in its colour, its most urgent upgrade, how many services are up to date, and a strip with
  one square per environment in its state there. Applications with four services or more take a double cell once a
  board has six or more. A board stays close to a square: five cells are three and two, not one row, and a long list
  takes more columns rather than growing tall. A small cell (a board fitted to a short screen) keeps its name, its
  environments and its state, and leaves out the small logo and the upgrade.
- **Open a cell** and its board is laid over the global one like a card, one cell per service; the global board stays
  behind, dimmed, with the application marked, and its cells open the other applications directly. Under its cells
  the card has the application's matrix: its services (by part) with the running versions in every environment.
  × or Escape lays the card down again. Arrow keys move between cells, Enter opens one.
- **Only what is not OK.** The legend counts the cells per state, on the board and above an opened card's cells; a
  state shows only its cells, and **not up to date** all but the current ones, so the few that need work in an
  application of hundreds of services stand alone. Again, or Escape, shows all.
- **Grouped by target**, each cell is a cluster or cloud account (`/tiles?group=target`): open one to see what runs
  on it, `/tiles?group=target&app=k8s-prod&only=1&tv=1` puts one cluster on a wall screen.
- **Only this team** (or application) on the card shows it alone, as the whole board: its services as large cells,
  nothing else behind (`/tiles?group=team&app=payments&only=1`). With *Env* and **TV mode** it is a team's own wall
  screen (`&env=prod&tv=1`); save it as a [view](#saved-views) to keep it a click away.

![An application's card laid over the global board: webshop's services as cells, the board behind dimmed with webshop marked](../../../assets/tiles-card.png) Open a service for its versions per environment, the newest release with its notes, its drift and a link to
  its page; Escape closes it. The arrow keys move between cells (up and down to the nearest cell above or below). Where a GitOps application deploys it, each environment behind offers **PR → version**
  (its [pull request](/guide/argocd/#upgrade-with-a-pull-request), previewed first), or links the one already open;
  when its newest release is unknown, it says [why](/guide/security/#newest-release-unknown).
- **Env** colours the board for one environment. **Side by side** draws one board per environment with every
  application in the same place, so you can see at a glance where prod lags behind staging.

![One board per environment, side by side: dev, staging and prod with the same applications in the same places](../../../assets/tiles-envs.png)

- The board follows changes live, and a cell whose state changed pulses. **TV mode** hides the header and fills the
  screen, for a wall display; **Cycle environments** there shows every environment in turn, then all of them side by
  side, 30 seconds each (`?tv=1&cycle=60` for another pace).
- The browser tab's icon is the logo coloured by the workspace: as many of its nine cells as needed turn warm or dim,
  so a pinned tab tells you when something needs attention.
- `/` opens your home view, on every device you sign in on: the view you switched to last, or the one picked under
  **Your account → Home page**. Until you pick, it goes by device: the tiles on a phone, the table elsewhere.

## Saved views

Set up the matrix or the tiles the way you look at them (a team's applications, production only, grouped by status,
the TV board for the wall screen) and **+ Save view** keeps it under a name. Saved views are tabs above the matrix and
the tiles: one click opens a team's board, and each has its own address to bookmark, share or open on a wall screen.

- **Yours or everyone's**: a view is yours unless a member saves it *For everyone*; then the whole workspace sees the
  tab, and members can delete it.
- **Your home**: *Make it my home* on a view's tab, or the home page setting on **Your account**, makes Goliash open
  with it on every device. Switching between table and tiles does not change a view home.
- **Search**: views are in the search (⌘K) too.
- **Updates** has views of its own (a team's upgrade list, production only), as tabs above it.

A view keeps the page's grouping, team or application, environment, filters and TV mode, not a point in time.

## One target at a glance

Click a target's name under **Settings → Agents and targets** to see what runs on that cluster or host: a ring
with the share of workloads that are up to date, counts of services, drift and workloads waiting in the inbox,
deploys per day over thirty days, and every workload as a card, one column per application. A card shows
the version (and a newer acceptable one), the image, a dot per running replica, drift badges and its service;
its colour says how current it is (green up to date, amber behind, red end of life or targets disagreeing, grey not
mapped), and problems come first. The filter above the cards matches names, services, versions and badges. Below,
**What changed here** lists the latest deploys on this target.

![A target page: a ring with the share of up-to-date workloads, deploys over 30 days, and workload cards grouped by application](../../../assets/target.png)

**Group by** arranges the columns, and the choice stays in the link:

| Group by | Columns |
|---|---|
| Application (default) | What the workload belongs to, from its labels (below). |
| Namespace | The Kubernetes or Nomad namespace, the Compose project or the Swarm stack. |
| Team | The owner of the workload's service; unmapped workloads and services without an owner go under *no owner*. |
| Status | Needs attention, behind, not mapped, up to date. |

A workload's application is the first of these labels it has: the workspace's own label key (set under
**Settings → Users → Applications**, for example `example.com/app`), `goliash.app`, `app.kubernetes.io/part-of`,
`app.kubernetes.io/instance` (the Helm release), `release`, `com.docker.compose.project` and
`com.docker.stack.namespace`. Without any of them, a Nomad job's name stands in, else its namespace.

Platforms such as Dokploy and Nomploy name what they deploy `<project>-<part>-<6 random characters>`. Goliash drops
the random part (`cefiro-db-wruzyw` is `cefiro-db`), and groups the parts of one project under its name: `cefiro`,
`cefiro-db` and `cefiro-redis` appear together as **cefiro**, with the parts listed under the heading. Names from
labels are never changed. The column caption says where the
name came from, so a column captioned *namespace, no app label* is a hint to label those workloads. A new label
key applies from each target's next snapshot.

## Looking back, and exporting for audits

![Time travel above the matrix: 6 October, a track from 12 September to now with a mark for each deployment, Play and step buttons, and the matrix as it was then with upstream and behind-staging badges](../../../assets/timeline.png)

Open **Time travel** above the matrix to scrub through the past: "what was in prod on 12 September at 14:00?". The
track marks every deployment, taller where more happened. Drag along it and the matrix redraws as you go, with the
versions that differ from the moment shown before sliding in and highlighted. **⏮ ⏭** (or ← → on the track) step from
deployment to deployment, Shift with the arrows moves a day, **Play** (or space) steps through the deployments one by
one up to now, and the chips jump a day, a week, 30 or 90 days back. **Compare with now** marks every cell of the
past that differs from today and says what runs there now: what changed since the incident, at a glance. While you scroll the matrix the track stays at the
top. The address follows, so a moment can be shared; **Exact moment** picks one to the minute. Goliash keeps when each version ran for 400 days, so the answer is right even after versions moved on and
back. The past is compared with the newest releases out by then (published by then, or first seen by Goliash when
the date is unknown), and its drift is worked out from what ran: a version behind that release, or behind the
environment before it. The **tiles** have the same timeline, coloured as things stood then.

**Export CSV** downloads every container that runs (or ran at that time), sidecars included, with environment,
target, workload, image, tag, digest, replicas and when it was first seen: an asset inventory for ISO 27001 or
SOC 2 audits. **SBOM** next to it downloads the same as a CycloneDX 1.6 document: one component per image, with an
OCI package URL and its digest, where it runs as `goliash:` properties (environment, target, service, workload), and
the packages of the image's SBOM nested inside where the registry holds one. Import it into Dependency-Track or hand
it to an auditor. The same is available from the CLI and the API:

```sh
goliash matrix -at 2026-09-12T14:00
goliash inventory -csv > inventory.csv
goliash inventory -format cyclonedx -environment prod > prod.cdx.json
goliash inventory -at 2026-09-12T14:00 -csv
curl -H "Authorization: Bearer $GOLIASH_TOKEN" "https://goliash.example.com/api/v1/inventory?format=csv&at=2026-09-12T14:00"
curl -H "Authorization: Bearer $GOLIASH_TOKEN" "https://goliash.example.com/api/v1/inventory?format=cyclonedx" > running.cdx.json
```

## Image hygiene

**Image hygiene** (linked above the matrix, with the number of warnings) lists running images that make "what
runs" hard to know or to trust, sidecars included:

| Finding | Meaning |
| --- | --- |
| moving tag | `latest`, `stable`, `main` or no tag: the version cannot be known, and a restart may pull something else. |
| retagged | The same repository and tag run as different images (digests): the tag was pushed again. |
| untrusted registry | The image comes from outside `GOLIASH_ALLOWED_REGISTRIES` (e.g. `ghcr.io/acme,docker.io/library`). |
| outdated base | Built on an older build of its `FROM` image than the base's tag has now (a build cache, no `--pull`): the base's fixes are missing. See [Security](/guide/security/#built-on-an-outdated-base). |
| over budget | Bigger than the **Image size budget** set in the service's version policy. |
| unexpected signer | Not signed by the **Expected signer** in the service's version policy: unsigned, signed by someone else, or signed with a key that names nobody. See [Security](/guide/security/#who-signed-it). |
| unpinned (info) | No digest is known for the image. |

`goliash hygiene`, `GET /api/v1/hygiene` and `goliash_image_hygiene_findings{kind}` in `/metrics` show the same.
Compose files are left out: they declare tags, not what runs.

## Services, environments and targets

- A **target** is one thing a collector reads: a Kubernetes cluster, an ECS region, a Nomad region, a Swarm or a
  Docker host. Each target belongs to one environment.
- An **environment** has a position that sets the promotion order: lower comes first (dev 10, staging 20, prod 30).
- A **service** is what you deploy, wherever it runs. Running workloads are mapped to services.

A workload can override its environment with the label `goliash.env`, for example when one cluster hosts both
staging and prod namespaces.

A service that runs nowhere any more can be deleted from its page by an admin, with its policy, releases, mapping
rules and acknowledgements; its history stays. Admins rename and reorder environments on the Agents page (drift is re-evaluated with the new order), and delete one
once no target belongs to it. **Edit** on a target changes its environment, settings and poll interval; its
collector picks the change up with the next configuration poll.

## Software Goliash does not collect

A database on a virtual machine, a managed service, an appliance, or a tool you are about to deploy: **Add by hand**
on the matrix (members) takes a name, the image repository its releases are read from (for example
`docker.io/library/postgres`) and, optionally, where it runs: an environment, a place (`db-vm-1`) and a version.

- **With a version** it runs on a *manual* target named after the place, and counts like anything collected: the
  matrix, drift, the Updates page, end of life, the [security posture](../security/) and notifications. On the
  service page, *Runs outside Goliash* lists the entries; enter a new version under the same place to record an
  upgrade, or remove it. Manual targets never go stale and are never given to an agent.
- **Without a version** it is watched: Goliash checks its releases, notifies `new_release` and lists it under
  *Watching* on the Updates page.

## How workloads map to services

Each running workload goes through these steps, and the first match wins:

1. **Ignore rules.** Images or containers you chose to ignore. A workload labelled (or tagged)
   `goliash.ignore=true` does not even reach Goliash: its agent leaves it out.
2. **Labels.** `goliash.service`, else `app.kubernetes.io/name`. The service is created when it does not exist yet.
3. **Rules.** Lowest priority number first. The inbox creates workload-name rules at priority 50 and image rules at
   100, so a rule for one workload wins over a rule for its image. `goliash rule create` defaults to 100.
4. **The inbox.** Everything else waits there with a suggested name, taken from the image.

The labels are Kubernetes labels, ECS tags, Nomad job meta, or Swarm and Docker labels.

Within a workload, the **main container** carries the version. It is the container named by the `goliash.container`
label, else one named like the workload or service. Known sidecars such as `istio-proxy`, `linkerd-proxy`, `envoy`
or `datadog-agent` are skipped.

## The inbox

The inbox groups unmapped workloads by image.

**Map new images automatically** (the switch at the top, also offered when you connect the first agent) skips
the inbox: a workload of an image no rule or label maps goes to the service suggested for it, from
`goliash.service`, Kubernetes and Compose labels or the image name, as soon as it reports, and the image gets a
rule like **Map all** makes. Sidecars are left out. Turning it on maps what waits in the inbox the same way;
turning it off keeps what was mapped. Each automatic mapping is in the audit log as `auto-map`. Rename or merge
services afterwards as you like.

- **Map all** maps every workload running the image to a service at once and adds an image rule, so new
  workloads with that image map by themselves.
- **Map only this** is for an image that runs as several services, for example one image deployed as `web` and
  `worker`. It maps a single workload and adds a workload-name rule, which takes precedence over image rules.
- **Ignore this image** removes it from the inbox at once and stops tracking it from the next snapshot.

Rules can also be created from the CLI:

```sh
goliash rule create -match image_repo -pattern 'ghcr\.io/acme/pay.*' -service payments
goliash rule create -match workload_name -pattern 'worker' -service jobs -priority 50
goliash rule create -match label -pattern 'team=payments' -service payments
goliash rule create -match ignore -pattern 'docker\.io/library/busybox'
```

Patterns are regular expressions that must match the whole value. The inbox page lists every rule; deleting one
returns the workloads it matched to the inbox with the next snapshot, unless another rule or a label maps them.

## History

Every snapshot is compared with the previous one of the same target. The differences become events:

| Event | When |
| --- | --- |
| `deployed` | A workload appears. |
| `version_changed` | A workload runs a different tag. A changed digest under the same tag is noted as a retag. |
| `removed` | A workload disappears. |
| `new_release` | An upstream release newer than everything known appears. |
| `drift_detected`, `drift_resolved` | Drift is announced, or ends. |
| `agent_stale` | An agent stops sending heartbeats. |
| `gitops` | An Argo CD application or Flux Kustomization has been out of sync, degraded or missing for 10 minutes: out of sync with what Git pins, degraded with the version change and the Git diff just before it, and a **Rollback PR** button when one is on offer; see [Argo CD and Flux](/guide/argocd/). |
| `pull_request` | Goliash opened a pull request to upgrade, promote or roll back a service, asked by someone or by the version policy's `auto_pr`; once more when one has waited for review for more than three days, when its checks (CI) fail, and when it was merged an hour ago and its version does not run yet (with why it waits); see [Argo CD and Flux](/guide/argocd/#upgrade-with-a-pull-request). |
| `security_release` | A new upstream release fixes vulnerabilities, as its notes or the project's feed (php.net) say, and a service runs an older version; once per release. See [Security releases](/guide/notifications/#security-releases). |
| `watch` | A new release of software a [security watch](/guide/notifications/#security-watches) covers (php, openssl…); once per watch, service and release. |
| `deadline` | An upgrade nears its [remediation deadline](/guide/security/#remediation-deadlines) (three days before) or passed it; once each. |
| `security_brief` | Daily or on Mondays: new security releases of what runs, watches, exploited vulnerabilities, security upgrades waiting, fixes not deployed; nothing when there is nothing. See [Security brief](/guide/notifications/#security-brief). |
| `goliash_release` | A newer Goliash release is out (checked every 6 hours, once per release; off with `GOLIASH_UPDATE_CHECK=off`). When its release notes name security fixes (`fix(security)`, a Go vulnerability or a CVE), it says *with security fixes* and names them; so does the Status page. |

The first snapshot of a target is a baseline and creates no events. When a snapshot is incomplete, for example
because one namespace was forbidden, missing workloads are not treated as removed.

---

# Notifications
Source: /guide/notifications/

> Send new releases, drift and stale agents to Slack, Microsoft Teams, Google Chat, Discord, Telegram, ntfy, Grafana, webhooks, e-mail or browsers, instantly or as digests.

Notifications have two parts: **channels** (where messages go) and **rules** (which events go there, and how
often). Both are managed on the Notifications page or with the CLI.

![The Notifications page: channels (e-mail, Microsoft Teams, web push with two browsers, Slack), rules with their time zones and quiet hours, and recent deliveries with one held for quiet hours, one in a digest and one that gave up with the mail server's answer](../../../assets/notifications.png)

## Channels

| Type | Settings |
| --- | --- |
| Slack | An incoming webhook URL. Mattermost and Rocket.Chat incoming webhooks take the same messages. |
| Microsoft Teams | The URL of a Teams workflow: in the channel, **Workflows** → *Post to a channel when a webhook request is received*. |
| Google Chat | A space's webhook URL: in the space, **Apps & integrations** → **Webhooks**. |
| Discord | A channel webhook URL (Channel settings → Integrations → Webhooks). |
| Telegram | A bot token from @BotFather and the chat ID the bot posts to. |
| ntfy | A topic URL on ntfy.sh or your own server, e.g. `https://ntfy.sh/my-goliash`, and an optional access token. |
| Grafana | Grafana's URL and a service account token with `annotations:write`; every item becomes an annotation. |
| Webhook | A URL and an optional signing secret. |
| E-mail | Recipients, and optionally the channel's own mail server (see below). |
| Web push | Nothing: the channel is there already; each person adds their browsers (see below). |
| In Goliash | Nothing: messages only show under the bell (see below). |

```sh
goliash channel create -type slack -name ops -url https://hooks.slack.com/services/…
goliash channel create -type webhook -name ci -url https://example.com/goliash -secret s3cret
goliash channel create -type discord -name releases -url https://discord.com/api/webhooks/…
goliash channel create -type teams -name platform -url https://prod-00.westeurope.logic.azure.com/workflows/…
goliash channel create -type gchat -name sre -url "https://chat.googleapis.com/v1/spaces/…/messages?key=…&token=…"
goliash channel create -type telegram -name team -token 123456:ABC… -chat-id -1001234567890
goliash channel create -type ntfy -name phone -url https://ntfy.sh/my-goliash
goliash channel create -type email -name oncall -to oncall@example.com
goliash channel test -name ops
```

Channel URLs and secrets are encrypted at rest and never shown in full in the UI.

### E-mail

Mail goes through the server's relay (`GOLIASH_SMTP_*`, see [Configuration](/reference/configuration/#e-mail)), or
through the channel's own mail server: open **Own mail server** when adding the channel, or edit it later, and give
the server as `host:port`, the sender address, and a user name and password if it needs them. Encryption is TLS on port
465 and STARTTLS elsewhere unless you pick one; *None* is for a relay on the same host or network. The password is
encrypted with the other channel secrets and never shown again; leave it empty when editing to keep it. **Send test**
checks the whole path. Without a relay on the server, an e-mail channel needs its own.

```sh
GOLIASH_CHANNEL_SMTP_PASSWORD=… goliash channel create -type email -name oncall -to oncall@example.com \
  -smtp-addr smtp.example.com:587 -smtp-from goliash@example.com -smtp-username goliash
```

### Web push

Every workspace has one web push channel, **Browser notifications**, made by itself: desktop notifications in
Chrome, Edge, Firefox and Safari, and on phones. Press **Notify this browser** next to it in every browser that should
get them (the browser asks for permission once); **Stop on this browser** takes it off again. Anyone with access to
the workspace can add their own browsers; the channel shows how many are on it. It comes with a rule that sends right
away what someone should look at now: exploited or critical vulnerabilities, remediation deadlines, stuck rollouts
and services down, rollbacks and stale agents. Change that rule, or add more, as for any other channel.

A notification shows the item's kind, service and change, and opens Goliash on that service when clicked; digests
show the first lines. Browsers that unsubscribed or expired are forgotten at the next message.

- Goliash needs HTTPS (or `localhost`) for browsers to allow push; `GOLIASH_PUBLIC_URL` should be that address.
- On iPhone and iPad (iOS 16.4 or later), add Goliash to the home screen first (Share → Add to Home Screen), then
  open it from there and press **Notify this browser**.
- Apple's push service (iPhone, iPad, Safari) only accepts pushes from a server that says who it is: an https
  `GOLIASH_PUBLIC_URL` (or `GOLIASH_PUSH_SUBJECT=mailto:you@example.com`). Without one the Notifications page says
  so, and Apple devices get nothing while Chrome, Edge and Firefox do.
- **Send test** reports every browser: which got it, which did not and what its push service answered (for example
  `web.push.apple.com answered 403: BadJwtToken`), and which had unsubscribed.
- Messages are encrypted for each browser (RFC 8291) and signed with a key pair the server makes on first use and
  keeps with the other secrets (encrypted at rest with `GOLIASH_SECRET_KEY`). The server sends only to the push
  services of browsers (Google, Mozilla, Apple, Microsoft).

## Tickets in Jira, Shortcut, GitHub, GitLab and Linear

A **Jira**, **Shortcut**, **GitHub Issues**, **GitLab Issues** or **Linear** channel files tickets instead of
sending messages, so what Goliash finds lands in a team's backlog
and leaves it by itself:

- **One ticket per upgrade** of a service in an environment (and application): *[Goliash] payments-api @ prod:
  upgrade 1.4.2 → 1.6.0*. It says what runs and what to move to, and why: exploited vulnerabilities, security
  fixes, critical CVEs, end of life, the remediation deadline, whether it is reachable from outside and patches
  waiting at the platform. It also links the release notes and the service in Goliash.
- **Fields.** The priority comes from the urgency (an exploited vulnerability is *Highest*, one step higher when
  it is reachable from outside): Jira's priority field and Linear's priority. Shortcut, GitHub and GitLab get it
  as a label (`priority-highest`, `priority: highest`, `priority::highest`). The due date is the remediation
  deadline, as Jira's due date, Shortcut's deadline and GitLab's and Linear's due date; on GitHub it is in the
  body. Labels are `goliash` and, where it fits, `security` and `end-of-life` (Linear: none). A Jira project
  whose screen lacks priority or due date gets the issue without them.
- **Later alerts** about the same upgrade become comments on its ticket, not new tickets.
- **Closing.** Goliash closes the ticket once the upgrade is no longer to do: deployed (*Done: payments-api runs
  1.6.0 in prod*), or put off as an accepted risk. In Jira it takes a transition to a done status, preferring
  one named Done, Closed or Resolved. In Shortcut it moves the story to its workflow's done state, in Linear to
  the team's completed state. GitHub and GitLab issues are closed.
- **Where it shows.** Updates and the service page show each open ticket's key with a link.

Which upgrades get a ticket is up to the channel's rules: a rule for *remediation deadlines* and *vulnerabilities*
in production, filtered by owner, is a good start. Alerts about something with no upgrade to do file nothing.
**Create ticket** on an Updates row files one by hand (members); with several trackers, pick one there.

Settings. Every tracker takes a default destination and one per team (the services' owner,
`team-payments=PAY, team-data=DATA`). Tokens are encrypted at rest and never shown again.

| Tracker | Sign-in | Where tickets go |
| --- | --- | --- |
| Jira | Cloud: e-mail and API token; Data Center: personal access token (no e-mail) | project key; issue type (default *Task*) |
| Shortcut | API token | team by mention name (optional); story type (default *chore*) |
| GitHub Issues | token that may write issues; API address for GitHub Enterprise Server | repository `owner/name` |
| GitLab Issues | token with the `api` scope; address for self-managed GitLab | project `group/name` |
| Linear | personal API key | team by key or name |

**Send test** signs in and checks every project, team or repository, filing nothing.

## The bell

The bell at the top of every page lists what Goliash sent you in the last 30 days, newest first, like the
notifications of any app. Each message has its title, its first lines and a link to where it points: the upgrade
plan opens **Updates**, an alert about one service opens that service, the rest open **History**.

- The red count on the bell is how many you have not opened yet; each person has their own. Opening a message reads
  that one only, so nothing slips by unseen in a long list; **Mark all read** reads them all.
- Every rule counts: a message sent to Slack, e-mail and push shows once. To have messages only in Goliash, add an
  **In Goliash** channel and point rules at it.
- Installed as an app (**Install** in Chrome or Edge, **Add to Home Screen** on iPhone), the app icon carries the
  same count, and a web push updates it even with Goliash closed.
- The count updates live while a page is open; **Send plan now** shows up there too.
- With [Goli](/guide/goli/) on, an alert or an upgrade plan has **Ask Goli**: Goli opens with a question about it ready
  to send (what the alert means and what to do first, or which upgrades come first). Nothing is asked until you
  press **Ask**.

## How messages look

Every item carries its kind, as an emoji and a colour: a new release ✨, drift ⚠️, an end of life ⛔, a resolved
drift ✅, an update ⬆️ coloured by urgency, a silent agent 🔌. With `GOLIASH_PUBLIC_URL` set, messages link back:
each item opens the matrix filtered to its service, and a button opens Goliash (the Updates page for an upgrade plan).

- **Slack** gets Block Kit: a header and a count by kind for digests, one section per item with the versions
  (`1.5.0` → `1.6.0`), a *Release notes* button, and *Open in Goliash* at the end; up to 20 items, then a count.
- **Discord** gets one coloured embed per item (up to ten; longer digests stay a list).
- **Microsoft Teams** gets an Adaptive Card: the title and count, one container per item in the kind's colour
  (attention, warning, good) with the versions, target and team, a *Release notes* button, and *Open in Goliash*.
- **Google Chat** gets a card with the logo in its header, one line per item with its kind, the versions in colour and
  a button (release notes, or the service in Goliash), and *Open in Goliash* at the bottom.
- **E-mail** is HTML with a plain-text alternative, in the logo's colours, readable in any mail client.
- **Web push** shows the kind's emoji and the service as the title, and the change as the text.

![A digest e-mail: a navy header with the count by kind, one card per item with a coloured edge, the versions and a button back to Goliash](../../../assets/notify-email.png)

## Rules

A rule sends some event types to a channel, optionally only for some services, owners (teams), applications or
environments (**Production only** is the last environment, whatever it is called; `-production` on the command line),
or only for releases of at least a given size. Applications are matched by the names the matrix shows
(after renames and merges, see [Applications and teams](/guide/apps/)): a rule for `webshop` gets every service
running in it, and a drift only when it is in that application.

A deploy back to an older version (2.4.1 → 2.3.9) is a **rollback**: ⏪ *payments-api @ prod: rolled back from
2.4.1 to 2.3.9*. Rules for deploys get rollbacks too, and the event type `rollback` (*rollbacks* in the form) sends
only those, e.g. to the people on call:

```sh
goliash notify create -channel oncall -events rollback -envs prod -mode instant
```

A rule with the event type `rollout` (*stuck rollouts and services down*) announces a workload running two versions
side by side for more than half an hour (⏳ *traefik @ prod: rollout stuck on k8s-prod: v3.1.6 on 1 of 2, v3.1.2 on
1*) or none of the replicas it asks for (🛑, urgent on web push), once each.

A rule that sends **stale agents** (an agent silent for 10 minutes) also sends the all clear when the agent is back:
✅ *agent eu-cluster is back after about 25 min; its targets report again*.

```sh
goliash notify create -channel ops -events new_release,drift_detected,agent_stale -mode daily -min-jump minor
goliash notify create -channel oncall -events drift_detected -envs prod -mode instant
```

| Mode | Delivery |
| --- | --- |
| `instant` | Within seconds. Items arriving together are sent as one message. |
| `daily` | A digest every day at the digest hour (**Digest at**, default 08:00). |
| `weekly` | A digest every Monday at the digest hour. |

The digest hour is in the rule's **time zone** (any IANA name, such as `Europe/Bratislava`; UTC when empty), so
08:00 stays 08:00 across daylight saving time.

**Quiet hours** keep right-away rules from waking people: what comes in during them is held and sent together, as one
message, when they end (22:00–07:00 overnight, or 09:00–17:00 for a channel only read after work). Combine them with a
web push channel for a phone that buzzes only by day.

```sh
goliash notify create -channel phones -events drift_detected,agent_stale -timezone Europe/Bratislava -quiet 22-7
goliash notify create -channel ops -mode daily -digest-hour 9 -timezone America/New_York
```

Admins **edit** a channel (secrets are never shown again; an empty field keeps the stored value). Members **edit** a
rule (its channel, events, mode, digest hour and filters), **pause** it (it queues nothing until resumed) or delete
it on the Notifications page; admins delete a channel together with its rules. An edit applies to what comes next:
notifications already queued keep their channel and time.

A release is announced once per service and version. Failed deliveries are retried with backoff, after 1, 2, 4, 8 and
16 minutes, then given up; see [Recent deliveries](#recent-deliveries). Acknowledged releases and drift are not sent; see
[Acknowledging](/guide/versions/#acknowledging).

## Recent deliveries

The bottom of the Notifications page lists the last 25 notifications and how each went: **sent**, waiting **in the
digest** until its hour, **attempt N failed** with the error and when it is tried next (after 1, 2, 4, 8 and 16 minutes),
or **gave up after 6 attempts**. **Retry now** sends a failed one at once and says whether it went through; **Send
now** sends one waiting for its digest. A wrong webhook URL or a revoked Slack hook shows here with what the
service answered.

## Upgrade plan

A rule with the event type `updates_plan` (*upgrade plan* in the form) sends the [Updates](/guide/versions/#updates)
list at its digest time: every day for a daily rule, on Mondays for a weekly (or instant) one. It is one message, most
urgent first: what runs, the version to move to, why, and the release notes; an upgrade that can open as a
[pull request](/guide/argocd/#upgrade-with-a-pull-request) links its preview (*Open PR → 1.7.0*). Updates put off with an acknowledgement
are left out, and the rule's services, owners and environments apply, so each team can get its own plan:

```sh
goliash notify create -channel team-payments -events updates_plan -owners team-payments -mode weekly
```

**Send plan now** on the rule sends its plan at once, to check the channel and the filters without waiting for
Monday; the scheduled plans are not affected.

With [Goli](/guide/goli/) on and `GOLIASH_AI_PLAN_SUMMARY=true`, the plan opens with **Goli's take**: a few sentences
on what to upgrade first and why, written from the plan's own lines (one model call per plan and channel, no other
data). When the model cannot be reached, the plan goes without it.

With `GOLIASH_AI_DIGEST_SUMMARY=true`, daily and weekly digests of three or more events open with Goli's take too:
what in them needs someone's attention and what is routine, or that nothing does.

> Goli's take: Upgrade payments-api in prod first: its log4j-core is exploited in the wild and 2.16.0 fixes it.
> Postgres 15 reaches end of life next month; the minor updates in staging can wait.

## Security posture report

A rule with the event type `posture_report` (*security posture report* in the form) sends the
[Security page](/guide/security/) as one message at the same times as the plan: daily, or on Mondays. It is the
weekly line a security officer follows: how many production services run behind a newer release, the median and
90th percentile of days behind, how many are 30+ and 90+ days behind, end of life passed and near, accepted risk,
blind spots (with why newest releases are unknown: *2 not a version, 1 checked by agent*), the three longest exposures, the services built on an [outdated base](/guide/security/#built-on-an-outdated-base), the known vulnerabilities of production's images (exploited ones named) and their packages whose [licenses need a look](/guide/security/#licenses), how the week's upgrades went out (the
pull requests Goliash opened: merged, running, the median from merge to running, and those [merged but not
deployed](/guide/argocd/#upgrade-with-a-pull-request) with why), and the same numbers a week before. It links to the Security page and, with Goli on, offers **Ask Goli**.

```sh
goliash notify create -channel ciso -events posture_report -mode weekly
```

> 🛡️ Goliash security posture for Default<br>
> prod runs 24 services, 6 behind a newer release: median 41 days, 90th percentile 212; 4 behind 30+ days, 2 behind
> 90+. End of life: 1 passed, 1 soon. Accepted risk: 2. Longest: keycloak 24.0.5 → 26.0.0 (212 days); …

It covers production, and a rule for every event does not get it: ask for it by name. It ends with the **slowest
teams** (by the services' owner): those with upgrades past their [remediation
deadline](/guide/security/#remediation-deadlines) or more than 30 days behind (*Slowest teams: team-data (2 past
deadline, 41 days behind); team-pay (35 days behind).*). A rule **filtered by owner** sends that team its own week
instead, so each team hears how it patches:

> team-data in prod: 3 of 12 services behind a newer release, median 41 days; 1 past the remediation deadline; 4
> upgrades in 90 days, 12 days to patch (median).

```sh
goliash notify create -channel team-data -events posture_report,deadline -owners team-data -mode weekly
```

Its other filters do not apply. **Send report now** on the rule sends it at once. For trends over time, the same numbers are
[metrics](/guide/observability/).

## Security brief

The posture report is for the week; the **security brief** is for the morning. A rule with the event type
`security_brief` (*security brief* in the form) sends, daily or on Mondays at the same time as the plan, what a
security team should look at today, in a few lines: releases found since yesterday that fix vulnerabilities in what
runs, vulnerabilities CISA listed as exploited since yesterday that run, the [security watches](#security-watches)
and how many services each covers, vulnerabilities exploited in the
wild that run (those past CISA's due date named), upgrades fixing vulnerabilities still waiting in Updates, those
past their [remediation deadline](/guide/security/#remediation-deadlines), and pull requests merged but not deployed. It links to the Security page.

```sh
goliash notify create -channel security -events security_brief -mode daily
```

> 🛡️ Security brief<br>
> 1 security release since yesterday: shop 8.3.36-fpm (8.3.34-fpm runs). Watching “php” (3 services). 2 vulnerabilities
> exploited in the wild run, 1 past CISA's due date (CVE-2026-1234). 4 upgrades fixing vulnerabilities wait in Updates.

When there is nothing to look at (watches alone are not news), nothing is sent. The same brief opens the
[Security page](/guide/security/) as *Since yesterday*. Like the report, it is asked for by name and covers the whole
workspace; **Send brief now** on the rule sends it at once (saying so when there is nothing new).

## Ask Goli about an alert

With [Goli](/guide/goli/) on, a single alert (a vulnerability, a drift, an upgrade, a silent agent) carries an
**Ask Goli** button in Slack, Teams and e-mail. It opens Goliash with Goli's panel and a question about that alert
ready, e.g. *Explain this vulnerability alert for us, what fixes it and how urgent it is*; pressing **Ask** sends it.
Opening the link alone asks nothing, so link previews and stray clicks cost nothing.

## Vulnerabilities

**Without setting anything up**, every workspace hears of serious problems in what runs: the first time Goliash
starts with it, it adds a rule that sends `vulnerability` and `security_release` events in production to
[the bell](#the-bell) (its own bell channel, *Goliash bell*). Change it or delete it like any rule; a deleted one is
not added again. While no rule sends vulnerabilities outside Goliash (Slack, Teams, e-mail…), the
[Security page](/guide/security/) says so: a critical vulnerability would wait for someone to open Goliash.

The `vulnerability` event announces a vulnerability exploited in the wild (CISA KEV) or rated critical as soon as it
runs somewhere, once per service and environment, with the package and the version that fixes it:

> payments-api @ prod: CVE-2021-44228 (exploited, ransomware, critical) in log4j-core 2.14.1, fixed in 2.15.0

When CISA's due date for fixing an exploited vulnerability ([BOD 22-01](https://www.cisa.gov/binding-operational-directive-22-01))
passes while it still runs, it is announced once more, *past CISA's due date*; one that is already past due when it
starts running says so in its first alert.

```sh
goliash notify create -channel security -events vulnerability -envs prod -mode instant
```

### Security releases

A vulnerability is often fixed before scanners know it: the project ships a release and says so, and its CVE
reaches OSV and the distributions hours or days later. The `security_release` event announces such a release as
soon as Goliash sees it, to every service running an older version:

> php: 8.3.36-fpm fixes vulnerabilities; 8.3.34-fpm runs

Goliash reads it from the release's notes on GitHub or GitLab (a CVE, GHSA or GO identifier, *security
release/fix*, or nginx's *Security:* lines), from the release's own notes page (PostgreSQL's names the CVEs it fixes)
and, for projects that say it elsewhere, from the project's own word:

| Software | Where Goliash reads it |
|---|---|
| PHP | [php.net's releases](https://www.php.net/releases/), which tag security releases (`security: php.net`) |
| Go | [Go's release history](https://go.dev/doc/devel/release): *includes security fixes to …* (`security: go.dev`) |
| Python, Ruby | a release cycle in *security fixes only* support on [endoflife.date](https://endoflife.date): every release of it (`security: eol-phase`) |
| Node.js, Redis, nginx, … | their GitHub release notes |
| PostgreSQL | each release's notes page |

The `security` setting comes from the catalog for these images, or from a service's version policy. On the Updates
page the upgrade becomes a **security fix**, right after exploited vulnerabilities, with what it fixes.

```sh
goliash notify create -channel security -events security_release,vulnerability -mode instant
```

### Security watches

Projects often warn before they publish (*PHP: two critical vulnerabilities, fixed on Oct 21, 14:00 UTC*). On the
[Security page](/guide/security/#security-watches), **Security watches** takes what to watch (`php`, `openssl`, an
image or a service), why and for how long, and lists at once what runs it and why (*runs docker.io/library/php*,
*package php8.2-cli*). Updates marks those upgrades *watch: php*, and the `watch` event announces their next release
the minute Goliash sees it:

> shop: 8.3.36-fpm is out, 8.3.34-fpm runs (security watch “php”) — two critical fixes on Oct 21

```sh
goliash notify create -channel security -events watch -mode instant
```

The first time a workspace's SBOMs are looked up completely, what already runs is recorded without announcing, so
an installation does not hear about every vulnerability it runs on day one; the Security page lists those. See
[Security posture](../security/#is-it-running).

## Image alerts

Six events watch the running images, each once per image and environment (rules for every event get them too,
and Goli can explain each):

| Event | When |
| --- | --- |
| `image_grew` (*image growth*) | A deploy's image is at least a quarter and 20 MB bigger than the one it replaced: a heavier base, a dependency pulled in by accident, build tools left in the final image. |
| `image_over_budget` | A deploy's image is bigger than the **Image size budget** in its service's version policy. |
| `outdated_base` | An image is built on a build of its base that has had a newer one for over two weeks: [rebuild with `--pull`](../security/#built-on-an-outdated-base). |
| `evidence_lost` | A deploy's image has no signature, SBOM or provenance though the one it replaced had: the build or its CI changed. |
| `license` (*AGPL or source-available packages*) | A deploy's image brings an application package under a network copyleft (AGPL, SSPL) or source-available license (BUSL, Elastic) that the image it replaced did not have, as its SBOM declares; see [Licenses](../security/#licenses). |
| `unexpected_signer` (*not signed by the expected signer*) | A deploy's image is not signed by the **Expected signer** in its service's version policy: unsigned, signed by someone else, or signed with a key that names nobody; see [Who signed it](../security/#who-signed-it). |

> api @ prod: image grew from 90 MB to 240 MB (+150 MB, +166% with 2.0.0; biggest new layer 145 MB: RUN apt-get install -y chromium)<br>
> api @ prod: 2.0.0 image is 240 MB, over its 200 MB budget<br>
> api @ prod: 2.0.0 has no signature, SBOM, though 1.1.0 had<br>
> api @ prod: 1.1.0 brings com.itextpdf/itextpdf 5.5.13.3 (AGPL-3.0-only)

```sh
goliash notify create -channel platform -events image_grew,image_over_budget,outdated_base,evidence_lost -envs prod -mode instant
```

Deploy alerts look at deploys of the last week, so turning a rule on does not announce old news. Sizes and evidence
are read from the registry with the [supply-chain evidence](../security/#supply-chain-evidence); the service page
charts the size per deploy.

## Deploys on your dashboards

A Grafana channel turns events into annotations, so a deploy shows up on the graphs it may have changed:

```sh
goliash channel create -type grafana -name dashboards -url https://grafana.example.com -token glsa_…
goliash notify create -channel dashboards -events deployed,version_changed,removed -envs prod -mode instant
```

Annotations are tagged `goliash`, the event type, the service and the environment. In a dashboard, add an
annotation query on the built-in Grafana data source, filtered by tags (`goliash`, and for example `prod`).

## What changed before an incident

The History page has a *changed in the last hour / 6 hours / 24 hours / 7 days* filter, and the same works from
the command line and the API:

```sh
goliash events -env prod -since 2h
curl -H "Authorization: Bearer $GOLIASH_TOKEN" "https://goliash.example.com/api/v1/events?environment=prod&since=2h"
```

## Webhook payload

```json
{
  "workspace": "Default",
  "digest": false,
  "link": "https://goliash.example.com",
  "items": [{
    "type": "new_release",
    "service": "payments-api",
    "owner": "team-payments",
    "from": "1.6.0",
    "to": "1.7.0",
    "note": "minor",
    "at": "2026-10-03T08:00:00Z",
    "text": "payments-api: new release 1.7.0 (minor), running 1.6.0",
    "url": "https://github.com/acme/payments-api/releases/tag/v1.7.0"
  }]
}
```

`environment`, `target` and `url` are present when they apply; `link` is the server's public URL. With a secret, the request carries
`X-Goliash-Timestamp` and `X-Goliash-Signature: sha256=<hex>`, where the hex is
`HMAC-SHA256(secret, timestamp + "." + body)`. Check the signature and reject old timestamps:

```python

def verify(secret: bytes, timestamp: str, body: bytes, signature: str) -> bool:
    if abs(time.time() - int(timestamp)) > 300:
        return False
    expected = hmac.new(secret, timestamp.encode() + b"." + body, hashlib.sha256).hexdigest()
    return hmac.compare_digest("sha256=" + expected, signature)
```

## Metrics instead of messages

If you alert from Prometheus, scrape `GET /metrics` with an API token instead:

| Metric | Labels | Value |
| --- | --- | --- |
| `goliash_deployed_version_info` | service, environment, version | running replicas |
| `goliash_outdated` | service, environment | 1 when behind upstream by the tracked jump |
| `goliash_drift_days` | service, environment, kind (and app, for a service compared per application) | days a drift has been open |
| `goliash_deploys` | service, environment | versions that arrived in the last 30 days |
| `goliash_lead_time_seconds` | service, from, to | median time a version took to the next environment, last 30 days |
| `goliash_image_hygiene_findings` | kind | images with a moving tag, a tag pushed again, an untrusted registry or no digest |
| `goliash_exposure_days` | service, environment (and app) | days a service has run behind an available release |
| `goliash_exposure_median_days`, `goliash_exposure_p90_days` | environment (production) | median and 90th percentile of those days in production |
| `goliash_exposures` | environment, over_days (30, 90) | production services behind an available release that long |
| `goliash_end_of_life` | environment, state (passed, soon) | production services on a release cycle past or near its end of life |
| `goliash_accepted_risk` | environment | production exposures quieted by an acknowledgement |
| `goliash_build_info` | version | 1 |
| `goliash_leader` | | 1 on the server that runs the background work |
| `goliash_snapshots_pending` | | snapshots received and not processed yet |
| `goliash_notifications_queued`, `goliash_notifications_failing` | | notifications not sent yet, and those that failed at least once |
| `goliash_agents` | status | agents online, stale, never connected or revoked |

Alerts worth having on Goliash itself: `goliash_agents{status="stale"} > 0`, `goliash_snapshots_pending > 20` for
ten minutes, `goliash_notifications_failing > 0`, and `sum(goliash_leader) != 1` across servers.

```yaml
scrape_configs:
  - job_name: goliash
    scheme: https
    authorization: { credentials_file: /etc/prometheus/goliash-token }
    static_configs: [{ targets: ["goliash.example.com"] }]
```

Ready-made dashboards and deploy markers on your graphs are on [Grafana dashboard](../grafana/) and
[SigNoz dashboard](../signoz/); more alerts are in [Metrics and alerts](../observability/#alerts).

## On your desk

[Goli desk bot](/guide/goli-bot/) is a 3D-printable robot of the Goliash mascot that rings a bell when Goliash sends
a notification.

---

# Metrics and alerts
Source: /guide/observability/

> Goliash /metrics for Prometheus and the OpenTelemetry Collector: what runs where, drift, delivery speed and security posture, ready-made Grafana and SigNoz dashboards, and alert rules.

For a quick look without any of that, **Settings → Status** (also the *Live* sign above the matrix) says whether
Goliash itself works and its data is fresh: agents and targets reporting, upstream releases checked, private
registries accepting their credentials, SBOMs, OSV and CISA's catalog current, notifications going out, and whether a
newer Goliash is out. Each line
names what is wrong and links to where to fix it. The same is `GET /api/v1/status` for an uptime monitor (alert when
`state` is `bad`) and the `status` tool for AI assistants.

Goliash fits next to the observability stack you already have in three ways:

1. **Metrics.** `/metrics` serves what runs where, drift, delivery speed, image hygiene and Goliash's own health in
   the Prometheus text format. Prometheus, Grafana Agent/Alloy, VictoriaMetrics and the OpenTelemetry Collector
   (for SigNoz, Grafana Cloud, Datadog…) all read it.
2. **Dashboards.** Ready-made dashboards read those metrics:
   - [Grafana dashboard](../grafana/): Prometheus scrape settings, the dashboard, deploy markers on your graphs and
     the Helm chart's ServiceMonitor, PrometheusRule and dashboard ConfigMap.
   - [SigNoz dashboard](../signoz/): the OpenTelemetry Collector receiver and the dashboard.
3. **Deploys on your graphs.** A deploy shows up as a marker on the graphs of the services it may have changed, from
   the metrics alone or from a Grafana notification channel; see [Grafana dashboard](../grafana/#4-deploys-on-your-graphs).

The quickest way is **Settings → Integrations** in Goliash: it creates a viewer token with one click and shows the
scrape settings for Prometheus and the OpenTelemetry Collector, filled in for your server, next to download links
for both dashboards.

## A token for the scraper

`/metrics` needs an API token with the viewer role. Create one under **Users → API tokens**, or:

```sh
goliash token create -name metrics -role viewer
```

Save the `glsh_api_…` value in a file the scraper can read, e.g. `/etc/prometheus/goliash-token`, and send it as a
bearer token; the [Grafana](../grafana/#2-scrape-goliash) and [SigNoz](../signoz/#2-scrape-goliash-with-the-opentelemetry-collector)
pages show the scrape settings.

## Alerts

Ready-made Prometheus rules are in
[`deploy/prometheus/goliash-alerts.yaml`](https://github.com/pipozzz/goliash/blob/main/deploy/prometheus/goliash-alerts.yaml)
(also a download on the Integrations page): add the file to `rule_files` in `prometheus.yml`. They cover a missing
scrape, the background work not running exactly once, stale agents, a growing snapshot queue, failing notifications
drift open for more than two weeks, production on a release past its end of life, production behind an available
release for 90+ days, an image that grew by a quarter (and 20 MB) in a day, an image over its size budget, a
base with a newer build for over a month, an image that lost its signature, SBOM or provenance, an exploited vulnerability running past CISA's due date, a GitOps application out of sync or unhealthy for an hour, and merged pull requests that have not deployed in two hours. The same queries work in Grafana alerting and SigNoz alerts:

| Alert | Query |
|---|---|
| Behind upstream in prod | `sum(goliash_outdated{environment="prod"}) > 0` |
| Drift open for a week | `max(goliash_drift_days) > 7` |
| Production past end of life | `goliash_end_of_life{state="passed"} > 0` |
| Production patched slower than a month | `max(goliash_exposure_median_days) > 30` |
| A team patches slower than two weeks | `goliash_team_patch_median_days > 14` |
| A team has upgrades past their deadline | `goliash_team_past_deadline > 0` |
| Agent stopped reporting | `goliash_agents{status="stale"} > 0` |
| Goliash falls behind | `goliash_snapshots_pending > 20` for 10 minutes |
| Notifications failing | `goliash_notifications_failing > 0` |
| Background work not running exactly once | `sum(goliash_leader) != 1` |
| Exploited vulnerability past CISA's due date | `goliash_vulnerabilities_past_due > 0` |
| Rollout stuck or service down | `goliash_rollout_problem{kind=~"down|stuck"} == 1` for 10 minutes |
| GitOps application out of sync or unhealthy | `goliash_gitops_trouble == 1` for an hour |
| Merged pull requests not deployed | `goliash_pull_requests{state="undeployed"} > 0` for 2 hours |
| Image grew a lot | `goliash_image_size_bytes > 1.25 * (goliash_image_size_bytes offset 1d)` |
| Image over its size budget | `goliash_image_size_bytes > on (service) group_left max by (service) (goliash_image_size_budget_bytes)` |
| Base outdated for a month | `goliash_image_base_behind_days > 30` |
| Image lost its signature, SBOM or provenance | `goliash_image_evidence == 0 and (goliash_image_evidence offset 1d) == 1` |

Alerts about versions themselves (new releases, drift, removed services) are better sent by Goliash directly; see
[Notifications](../notifications/).

On Kubernetes, the Helm chart can create a ServiceMonitor and a PrometheusRule with these alerts; see
[Grafana dashboard](../grafana/#kubernetes).

## Metrics

| Metric | Labels | Value |
|---|---|---|
| `goliash_deployed_version_info` | service, environment, version | running replicas |
| `goliash_outdated` | service, environment | 1 when behind upstream by the tracked jump |
| `goliash_drift_days` | service, environment, kind (and app) | how long a drift has been open, in days |
| `goliash_deploys` | service, environment | versions that arrived in the last 30 days |
| `goliash_rollbacks` | service, environment | changes back to an older version in the last 30 days |
| `goliash_lead_time_seconds` | service, from, to | median time a version took to the next environment, last 30 days |
| `goliash_license_packages` | class, kind (app, os) | packages in running images per license class, from their SBOMs |
| `goliash_vulnerabilities` | environment, severity, exploited | known vulnerabilities in running images, from SBOMs and OSV |
| `goliash_vulnerabilities_past_due` | environment | exploited vulnerabilities running past CISA's due date for fixing them |
| `goliash_exploited_vulnerability_days` | environment | days the longest-running exploited vulnerability has run |
| `goliash_rollout_issues` | kind (down, stuck, short) | workloads whose replicas are not where they should be |
| `goliash_rollout_problem` | service, environment, target, workload, kind, running, desired | 1 for each such workload, while it lasts |
| `goliash_gitops_applications` | tool (argocd, flux, git) | GitOps applications deploying workloads |
| `goliash_gitops_trouble` | service, environment, target, app, tool, trouble | 1 for each application out of sync, degraded or missing, while it lasts |
| `goliash_gitops_sync_blocked` | service, environment, target, app, tool | 1 for each application an Argo CD sync window holds or Flux has suspended |
| `goliash_gitops_manual_sync` | service, environment, target, app, tool | 1 for each Argo CD application without an automated sync policy |
| `goliash_pull_requests` | state (open, merged, closed, undeployed) | pull requests Goliash opened in the last 30 days; undeployed: merged, version not running yet |
| `goliash_pull_request_to_running_seconds` | | median time from opening a pull request to its version running, last 30 days |
| `goliash_merge_to_running_seconds` | | median time from merging it to its version running, last 30 days |
| `goliash_image_hygiene_findings` | kind | images with a moving tag, a tag pushed again, an untrusted registry, an outdated base, a size over budget or no digest |
| `goliash_image_size_bytes` | service, environment | what pulling the service's image downloads, in bytes (compressed layers and config) |
| `goliash_image_layers` | service, environment | layers in that image |
| `goliash_image_size_budget_bytes` | service | the size budget from the service's version policy (`max_size_mb`), in bytes |
| `goliash_image_base_behind_days` | service, environment | days the image's base tag has had a newer build than the one the image is built on (0: current) |
| `goliash_image_evidence` | service, environment, kind (signature, sbom, provenance) | 1 when every image running there has it, 0 when one lacks it |
| `goliash_exposure_days` | service, environment (and app) | days a service has run behind an available release |
| `goliash_exposure_median_days`, `goliash_exposure_p90_days` | environment (production) | median and 90th percentile of those days in production |
| `goliash_team_behind`, `goliash_team_behind_median_days` | owner, environment (production) | a team's production services behind a newer release, and the median days they have been |
| `goliash_team_past_deadline` | owner, environment | a team's upgrades past their [remediation deadline](/guide/security/#remediation-deadlines) |
| `goliash_team_patch_median_days` | owner, environment | median days a team's upgrades deployed in the last 90 days had run behind (its time to patch) |
| `goliash_exposures` | environment, over_days (30, 90) | production services behind an available release that long |
| `goliash_end_of_life` | environment, state (passed, soon) | production services on a release cycle past or near its end of life |
| `goliash_accepted_risk` | environment | production exposures quieted by an acknowledgement |
| `goliash_build_info` | version | 1 |
| `goliash_leader` | | 1 on the server that runs the background work |
| `goliash_snapshots_pending` | | snapshots received and not processed yet |
| `goliash_notifications_queued`, `goliash_notifications_failing` | | notifications not sent yet, and those that failed at least once |
| `goliash_agents` | status | agents online, stale, never connected or revoked |

---

# SBOM of what runs
Source: /guide/sbom/

> A CycloneDX 1.6 software bill of materials of everything that actually runs, per environment and at any point in time, with each image's packages, for Dependency-Track, auditors and the Cyber Resilience Act.

A **software bill of materials** (SBOM) lists what a piece of software is made of: every component with its exact
version, a package URL that identifies it (`pkg:deb/debian/openssl@3.0.15-1~deb12u1`) and a hash. Security tools
match it against vulnerability feeds; auditors and customers ask for it; the EU Cyber Resilience Act, NIS2 and the
US Executive Order 14028 make it the expected answer to "what software do you run?".

Most SBOMs describe **one image at build time**. Goliash gives you the SBOM of **what actually runs**: every image in
every environment, read from the runtime, with the packages of each image nested inside, for now or for any moment
of the last 400 days.

## What is in it

One [CycloneDX 1.6](https://cyclonedx.org/) JSON document, validated against the official schema:

- **One component per running image**, type `container`, with its OCI package URL and SHA-256 digest, so the
  component is the exact build that runs and not a tag that may have moved.
- **Where it runs**, as properties: `goliash:environment`, `goliash:target`, `goliash:platform`, `goliash:service`
  and `goliash:workload`. An image that runs in three clusters lists all three.
- **Its packages**, nested inside the image, when its registry holds an SBOM attestation (see
  [Where the packages come from](#where-the-packages-come-from)): Debian, Alpine, Wolfi and RPM packages, npm, PyPI, Maven,
  Go modules and whatever else the build's SBOM lists, each with its package URL and, when the SBOM names it, its
  license (`MIT`, or an SPDX expression such as `GPL-2.0-or-later OR MIT`).
- **Servers watched by the [host collector](../../reference/collectors/#linux-servers)**: the operating system as an
  `operating-system` component and the packages Goliash follows as `application` components.
- **Software added by hand** (a managed database, an appliance) as `application` components.

```json
{
  "bomFormat": "CycloneDX",
  "specVersion": "1.6",
  "metadata": {
    "timestamp": "2026-10-12T09:30:00Z",
    "tools": { "components": [{ "type": "application", "name": "goliash", "version": "1.31.0" }] },
    "component": { "type": "application", "name": "Default: running software" }
  },
  "components": [
    {
      "type": "container",
      "name": "ghcr.io/acme/payments-api",
      "version": "2.14.1",
      "purl": "pkg:oci/payments-api@sha256%3A9f2c…?repository_url=ghcr.io%2Facme%2Fpayments-api&tag=2.14.1",
      "hashes": [{ "alg": "SHA-256", "content": "9f2c…" }],
      "properties": [
        { "name": "goliash:environment", "value": "prod" },
        { "name": "goliash:target", "value": "eks-prod-eu" },
        { "name": "goliash:service", "value": "payments-api" },
        { "name": "goliash:workload", "value": "payments/payments-api" }
      ],
      "components": [
        { "type": "library", "name": "debian/openssl", "version": "3.0.15-1~deb12u1", "purl": "pkg:deb/debian/openssl@3.0.15-1~deb12u1?arch=amd64" },
        { "type": "library", "name": "org.apache.logging.log4j/log4j-core", "version": "2.14.1", "purl": "pkg:maven/org.apache.logging.log4j/log4j-core@2.14.1" }
      ]
    }
  ]
}
```

## Export it

| Where | What you get |
| --- | --- |
| **SBOM** above the matrix | Everything that runs now, or at the moment [time travel](../matrix/#looking-back-and-exporting-for-audits) shows |
| **Security → Export SBOM** | Everything that runs now |
| `GET /api/v1/inventory?format=cyclonedx` | The same over the [REST API](../../reference/api/); add `environment=prod` for one environment and `at=2026-09-12T14:00` for a moment in the past |
| `goliash inventory -format cyclonedx` | The same from the CLI, with `-environment` and `-at` |

```sh
# what ran in production on 12 September at 14:00, for the auditor
curl -H "Authorization: Bearer $GOLIASH_TOKEN" \
  "https://goliash.example.com/api/v1/inventory?format=cyclonedx&environment=prod&at=2026-09-12T14:00" \
  > prod-2026-09-12.cdx.json
```

A viewer API token is enough: it reads only.

## Dependency-Track

[Dependency-Track](https://dependencytrack.org/) watches SBOMs against its vulnerability feeds and your policies.
**Settings → Integrations → Dependency-Track** creates a viewer token and gives a script that uploads one project
per environment, `goliash dev`, `goliash staging`, `goliash prod`, each with version `running`, creating them on the
first upload. Run it every hour from cron or a scheduled CI job and Dependency-Track always tracks what runs, not
what was built.

The Dependency-Track API key needs `BOM_UPLOAD` and `PROJECT_CREATION_UPLOAD`. The default *Automation* team has the
first but not the second; without it the upload answers 401.

## Where the packages come from

Goliash does not scan images. It reads the SBOM the build already attached to the image in its registry, SPDX or
CycloneDX, wherever it is:

- **BuildKit**: `docker buildx build --sbom=true`, or `sbom: true` in GitHub Actions' `docker/build-push-action`.
  That one flag is all an image needs.
- **cosign**: `cosign attest --type cyclonedx` or `--type spdxjson`, for example with an SBOM from
  [Syft](https://github.com/anchore/syft) or Trivy. Chainguard images come with one per platform; Goliash reads the
  linux/amd64 one when the image index's own SBOM only lists its platforms.
- **OCI referrers**: an SBOM pushed as an artifact (`oras attach --artifact-type application/vnd.cyclonedx+json`) or
  a Sigstore bundle with an SBOM predicate.

An image without an SBOM is not left empty: Goliash reads its operating system's package database (Debian,
Ubuntu, Alpine, Wolfi) and its application's manifests (npm, PyPI, Maven, Go, RubyGems, NuGet) from its layers
instead (see [Images without an SBOM](../security/#images-without-an-sbom)), so its packages are in the export too.
Images read before 1.177 are read again once, for their application packages. An [OpenVEX](https://openvex.dev) document attached the same ways (BuildKit
`--attest type=vex`, `cosign attest --type openvex`, a referrer) is read beside the SBOM: what the image's publisher
says does not affect it is left out of its vulnerabilities.

Images on public registries are read by the server. Images on private registries are read by the agents with the
credentials they already use, or by the server with read access given under
[Settings → Private registries](../../reference/collectors/#credentials-on-the-server). Each image is read once; its
package list is kept with its digest.

**Security** shows how many production images carry an SBOM, a signature and provenance, and lists those without.
An image Goliash has no packages for (no SBOM, and no package database it can read: scratch, RPM-based, or a
registry it cannot reach) still appears in the export, with its digest and where it runs, just without packages
inside: add `--sbom=true` to its build and the next export has them.

## Vulnerabilities from the same packages

The packages behind the export also answer **Security → Is it running?**: Goliash asks
[OSV](https://osv.dev) daily which known vulnerabilities affect them, puts those CISA knows to be exploited first, and
names the version that fixes each, or the newer distribution release or upstream version that does. Findings that do
not apply (kernel packages, packages past upstream's fix, what the image's VEX rules out, what people mark not
affected) are left out with their reason. See [Is it running?](../security/#is-it-running) and
[Not affected](../security/#not-affected).

## For audits and regulation

- **Cyber Resilience Act and NIS2**: an up-to-date inventory of the software you operate, per environment, in a
  standard format, with history: what ran when an incident happened.
- **SOC 2 and ISO 27001**: asset inventory and change evidence; pair the SBOM with the CSV and PDF of the Security
  page.
- **Customer and vendor questionnaires**: hand over the SBOM of production instead of filling in component lists.

The export only reads what Goliash already knows; it sends nothing anywhere. Package names and versions go to OSV
for the vulnerability lookups only, never image, service or host names.

---

# Security posture
Source: /guide/security/

> How long production runs behind available releases, what is past its end of life, supply-chain findings and blind spots, with a CSV and PDF for risk registers and audits.

The **Security** page answers the questions a security officer asks about what runs, from the runtime itself rather
than from tickets or pipelines:

![The Security page: tiles for the median exposure of production, services behind for 30 or more days, end of life, accepted risk and blind spots, then each production service with its running version, the fix, and how many days it has been exposed](../../../assets/security.png)

- **How long has production been exposed?** For every service behind a newer release, the time since the *first*
  newer release came out. The median and the 90th percentile summarise it, with the count behind for more than 30
  and 90 days.
- **What is new today?** *Since yesterday* opens the page with the [security brief](../notifications/#security-brief):
  releases found since yesterday that fix vulnerabilities in what runs, vulnerabilities CISA newly lists as
  exploited, upgrades past their [remediation deadline](#remediation-deadlines), upgrades fixing vulnerabilities
  still waiting and fixes merged but not deployed, each a link to where to act.
- **What is past its end of life?** Release cycles whose support has ended, or ends within 60 days
  ([endoflife.date](https://endoflife.date)).
- **What risk has been accepted?** Upgrades quieted with an acknowledgement count as accepted risk, with who
  acknowledged them and until when (on the service page).
- **Can the images be trusted?** Moving tags, tags pushed again, registries outside the allowed list and images
  without a known digest, from [image hygiene](../matrix/#image-hygiene).
- **Where are we blind?** Targets that stopped reporting, targets that report no digests, and services whose newest
  release is unknown, so their exposure cannot be measured.

The tiles are filters: *Behind for 30+ days*, *Past end of life* and *Accepted risk* show only those exposures
(*Show all* lifts it, and it stays when you switch between production and every environment), *Blind spots* jumps to
its section. Where a GitOps application deploys an exposed service, **Open PR** proposes the fix next to it.

## Security watches

An advance notice names software, not your services. **Security watches** on the Security page turns it into a
list: type what to watch (`php`, `openssl`, `ghcr.io/acme/api`), why and for how long (a week to three months, or
until removed), and each watch shows the services that run it, with the version and why they match: their name,
an image or workload they run, a GitOps application deploying them, or a package in their images' SBOMs. Updates
marks their upgrades *watch: php*; the `watch` notification announces their next release at once. Members start and
end watches; everyone sees them.

## Remediation deadlines

How long may an exploited vulnerability run? **Remediation deadlines** on the Security page set it per workspace,
in days, for three reasons to upgrade:

| Reason | The clock starts |
|---|---|
| Exploited vulnerability | when CISA listed it while the image ran there (or when the image started running, if later) |
| Security fix | when Goliash found the first release on the way that fixes vulnerabilities |
| End of life | on the end-of-life date of the running release cycle |

An upgrade with several reasons takes the earliest deadline. The `deadline` notification announces each upgrade
three days before its deadline and again the day it passes, once each (*postgres @ prod: the remediation deadline
for the end of life passed on 23 Oct; 13.4 runs, 13.16 fixes it*); a rule filtered by owner sends a team its own:

```sh
goliash notify create -channel team-data -events deadline -owners team-data -mode instant
```

An upgrade past its deadline for more than a week is no longer announced; the brief and the report name it. [Updates](../versions/#updates) marks each upgrade *due in
3 days* (amber within a week) or *overdue 5 days* (red), and a *past deadline* tile lists only those; the CSV has
`due` and `days_left`, `GET /api/v1/updates` has `due`, `due_for` and `days_left`. The panel counts what is past and
what is due within a week, and the [security brief](../notifications/#security-brief) and the
[posture report](../notifications/#security-posture-report) name the upgrades past their deadline. Admins set the
deadlines (none, 3 days to half a year each); with none set, Updates ranks by urgency alone. An acknowledged upgrade
(accepted risk) is not counted as past its deadline.

### Accepted risks

Putting off an update (Updates → *Put off*, or *Quiet notifications* on a service page) accepts its risk for a
while. Say **why** in the reason field: the reason goes into the audit log, the Security page's *Accepted risks*
panel and the [monthly report](#for-audits).

For audits that ask who decided, an admin ticks **Accepted risks need an admin's approval** under the deadlines.
Then a member's put-off:

- needs a reason;
- ends within 90 days at the latest (one until a version ends then too, if the version is not out by then);
- quiets nothing, and is not counted as accepted, until an admin approves it in the *Accepted risks* panel, or turns
  it down.

An admin's own put-off is approved at once. The `deadline` notification asks for the approval (*postgres @ prod:
dev@example.com accepts the risk until 31 Oct and waits for an admin's approval: "behind the VPN only"*) and, three
days before an accepted risk with a reason ends, says so, once each. Through the API, `POST /api/v1/acks` takes
`reason` and answers `pending`.

## Newest release unknown

A service whose newest release Goliash does not know cannot be measured: it shows as *unknown* in the tiles and
among the blind spots. Each says why, and what would let Goliash tell, on the Security page, the service's page,
its tile and the matrix (hover the note):

| Why | What to do |
|---|---|
| *not a version*: it runs `latest`, a branch or a commit | deploy a version tag; a moving tag is resolved when the registry lists the same image under a version tag |
| *check failed*: the registry refused or does not know the repository | registry credentials (server or agents), or the [upstream](/guide/versions/) it is published to |
| *checked by agent*: a private registry no agent has reported yet | credentials for the agents (`GOLIASH_CREDENTIAL_…`), or for the server |
| *no comparable tag*: none of the registry's tags reads like the running one (`1.4-custom`) | a **tag filter** in the version policy |
| *none in pinned major*: the policy pins a major with no release | change or remove the pin |
| *no registry image*: an image built where it runs | the repository it is published to, as the service's upstream |
| *not checked yet* | it is checked within minutes; or **Check now** |

## Base images

A service's own image is only as supported as the image it is built on: `node:18-alpine` reached its end of life in
April 2025 whatever the application's version. Goliash reads the base from the
`org.opencontainers.image.base.name` annotation or label, which BuildKit (`docker buildx`) records, when it reads
the image's source repository. The service page shows it under **Built on**, with its end of life from
[endoflife.date](https://endoflife.date), and the Security page lists every running service **built on an
unsupported base**: past its end of life, or within 60 days of it.

An image that does not declare its base can name it in the service's version policy (**Base image**, for example
`node:22-alpine`); `none` turns tracking off. The image is read on a public registry by the server; for private
images, set the base in the policy.

### Built on an outdated base

A new build is not always a fresh one: a build cache, or `FROM` without `--pull`, keeps the base's layers as they
were when the base was first pulled, with the vulnerabilities its newer builds fixed. The image's version and date
look current; its bottom layers are not.

Goliash checks this with the layers themselves. Building `FROM node:22-alpine` copies the base's layers unchanged,
so an image built on the base as its tag is now starts with exactly those layers. When it reads a running image's
evidence, Goliash also reads the base's tag (for the same platform, once per check) and compares:

- **current**: the image starts with the base's layers as they are now;
- **outdated**: it does not: it was built on an older build of the base (or on another base than it declares).

Outdated ones show as **Outdated base** on [Image hygiene](/guide/matrix/#image-hygiene), with when the base's current build was
made (*built on an older node:22-alpine than the one its tag has now (built Oct 2, 14 days ago): rebuild with
`--pull`*), in the Security page's supply chain, as an **older base** badge on the service's images, and in
`goliash_image_hygiene_findings{kind="outdated-base"}`. Bases pinned to a digest (`node@sha256:…`) are pinned on
purpose and left alone. Agents send the layers of private images (agent 1.61); the base itself must be readable by
the server, as public bases are.

### Chains of base images

Teams often build on their own base: the application is `FROM registry.example.com/base-java:21`, and that is
`FROM eclipse-temurin:21-jre-alpine`. The application can be built on `base-java:21` as it is now while
`base-java:21` itself was never rebuilt on the newer temurin, so temurin's fixes reach neither. Goliash follows the
chain down, up to four images deep:

- each image's base comes from its `org.opencontainers.image.base.name` label, or else from its BuildKit
  provenance attestation (`--provenance`, on by default with `docker buildx build --push`), which names every image
  the build used: Goliash takes the one whose layers, as they were at build time, the image starts with (a builder
  stage's image is not it);
- each link is compared like the first: current or outdated against its tag now;
- the chain stops at an official Docker image (`docker.io/library/…`, rebuilt by their maintainers on their bases),
  at an image whose base Goliash cannot tell or read, and four images down.

An outdated link anywhere makes the image's base outdated, and says which image to rebuild first: *built on
base-java:21, which is built on an older eclipse-temurin:21-jre-alpine than the one its tag has now: rebuild
base-java:21 with `--pull`, then this image*. The service page draws the chain under each image
(*on base-java:21 (current) → eclipse-temurin:21-jre-alpine (outdated)*), the Security page lists a base past its
end of life anywhere down the chain (*eclipse-temurin:17 under base-java:17*), and `GET /api/v1/services/{name}/images`
has `base_chain` and `rebuild_first`. Private bases are read with the workspace's
[registry credentials](/reference/collectors/#credentials-on-the-server); without them the chain stops there.

## Supply-chain evidence

For every image running in production, Goliash asks its registry what it holds beside the image:

- **Signed**: a [cosign](https://docs.sigstore.dev/cosign/) signature (`sha256-<digest>.sig`), a Sigstore bundle or a
  Notation signature as an OCI referrer.
- **SBOM**: an SPDX or CycloneDX attestation, as `docker buildx build --sbom=true` attaches it, or an SBOM artifact.
- **Provenance**: a SLSA provenance attestation (`--provenance=true`, GitHub artifact attestations).

The Security page shows the share of production images with each, and lists the unsigned ones. Goliash records that
these **exist**; it does not verify the signatures. Images on public registries are looked up by the server, at
most 40 per workspace and check, again after a week when something was found and after a day when nothing was.
Images on private registries are looked up by the agents, with the credentials they already use for private tags
(up to 20 digests per repository and round), and they send the packages of the SBOM along; this needs agent 1.27 or
newer. Images on Amazon ECR need the agent's role to have `ecr:GetAuthorizationToken`, `ecr:BatchGetImage` and
`ecr:GetDownloadUrlForLayer` (the CloudFormation and Terraform setups include them). Images without a known
digest count as *not checked*.

The same lookup reads the image's **size**: what pulling it downloads, its compressed layers and config, as
registries show it (linux/amd64, or the first platform, of a multi-platform image). A service's page lists it per
environment, with how much each differs from production (*76 MB +32 MB*): what promoting that version adds, and
**Image size** charts it per environment over the last 90 days, from the deploy history: each step is a deploy, so
a heavier base or a new dependency stands out. The metric `goliash_image_size_bytes` and the dashboards' *Images*
section show it in Grafana and SigNoz, next to each service's size budget, outdated bases and missing evidence. Under the chart, **Last change** lists what the latest deploy (production first) changed in
the image's layers: the new ones and the gone ones, biggest first, with the Dockerfile step that made each, read from
the image config's history (`RUN apt-get install -y chromium`, +145 MB). Layers both versions share, the base and
unchanged steps, are left out: what is listed is what grew. Agents read it for private images from agent 1.61.
`goliash images SERVICE` shows the same in a terminal.

### Who signed it

A keyless signature (`cosign sign` in CI, GitHub artifact attestations, Chainguard and many official images) carries
a short-lived certificate from Sigstore's Fulcio naming who signed: the CI workflow
(`https://github.com/acme/api/.github/workflows/release.yml@refs/heads/main`), a service account or a person's email,
and the identity provider that vouched for it (GitHub Actions, GitLab, Google). Goliash reads it from cosign's
signature and from Sigstore bundles, for public images and, from agent 1.70, for private ones:

- each service's **Security** panel shows who signed each running image;
- the Security page lists **Signed by**: every signer of production images and how many images each signed, so a
  person's email among workflows stands out;
- a service's version policy can name its **expected signer**, exactly or as a prefix (`https://github.com/acme/*`);
  an image that is not signed by it (unsigned, signed by someone else, or signed with a key that names nobody)
  shows as *unexpected signer* on [Image hygiene](/guide/matrix/#image-hygiene).

Goliash reads the identity as the certificate states it; it does not check the certificate's chain to Fulcio or the
signature itself. Use `cosign verify --certificate-identity … --certificate-oidc-issuer …` (or a policy controller
in the cluster) to enforce it; Goliash shows whether what runs matches what you expect.

## Is it running?

**Security → Is it running?** answers "is CVE-2024-3094 in production?" in seconds, and lists every known
vulnerability of the running images:

![Known vulnerabilities, only those exploited in the wild: two Log4j CVEs marked exploited, ransomware and critical with CISA's due dates, in log4j-core 2.14.1 of payments-api in prod, fixed in 2.15.0 and 2.16.0](../../../assets/vulns.png)

1. For images whose registry holds an SBOM (a BuildKit or cosign attestation, or an OCI referrer, see
   [SBOM of what runs](../sbom/#where-the-packages-come-from)), Goliash reads its package list once: package URLs
   such as `pkg:deb/debian/openssl@3.0.11-1~deb12u2` or `pkg:npm/lodash@4.17.15`. Images without one are read from
   their package database ([Images without an SBOM](#images-without-an-sbom)).
2. It asks [OSV](https://osv.dev) which known vulnerabilities affect those packages, again every day, and reads
   each advisory once for its CVE aliases, summary and severity. Debian, Ubuntu and Alpine packages are asked about
   in their distribution release, Wolfi and Chainguard packages under their source package (Chainguard's own
   advisories), language packages (npm, PyPI, Go, Maven, crates, RubyGems, NuGet …) by their package URL.
3. Search by CVE or advisory ID, package or service. A vulnerability shows once, under its CVE, with every advisory
   that names it (`DEBIAN-CVE-…`, `DSA-…`, `GHSA-…`) and each package and service it is in.

**What to patch first.** Vulnerabilities in CISA's catalog of
[known exploited vulnerabilities](https://www.cisa.gov/known-exploited-vulnerabilities-catalog) (KEV) come first,
marked *exploited* with CISA's due date and *ransomware* when CISA knows of ransomware using them; *Only exploited
in the wild* shows just those. Each affected package says what fixes it: the version (the lowest above the running
one, from OSV); else a newer release of its distribution
([Fixed in a newer release](#fixed-in-a-newer-release-of-the-distribution)); else the version upstream fixed it in,
with the newest release to run ([Upstream fixes](#upstream-fixes)); and only when none of these, *no fix yet*.
What does not apply is left out, with its reason ([Not affected](#not-affected)). The catalog is read once a day; `GOLIASH_KEV_URL` names a mirror or turns it off.

**How long it has run.** Each vulnerability says how long it has been running (*running for 23 days*: since the
first affected image started), and an exploited one still running after the date CISA set for fixing it
([BOD 22-01](https://www.cisa.gov/binding-operational-directive-22-01)) is marked *past due*; the Security page counts
them. `GET /api/v1/vulnerabilities` has `running_since`, `exploited.past_due` and `since` per affected image.

![The Security panel of a service page: four running images, each signed with an SBOM and provenance, and its vulnerabilities, the two exploited Log4j CVEs first with the versions that fix them, then jackson-databind ones by severity](../../../assets/service-security.png)

Each service's page has a **Security** panel too: its running images per environment, signed or not, with an SBOM
and provenance or not, and the vulnerabilities in them, exploited ones first, each with the package and the version
that fixes it.

A cell of the matrix whose running image has a vulnerability exploited in the wild shows it as a red
*exploited CVE-…* badge (its tile needs attention), and [Updates](../versions/#updates) puts that upgrade first.

A notification rule with the `vulnerability` event announces each exploited or critical vulnerability as it starts
running ([Notifications](../notifications/#vulnerabilities)).

**Export CSV** on the vulnerabilities page downloads what it shows (all of it, not only the first 300), one row per
vulnerability and affected image, with CISA's exploited flag and due date, whether it is past due, the fix and since
when it runs there: for an audit, or to import into a ticket system. `GET /api/v1/vulnerabilities?format=csv` gives
the same. Cells that start like a formula are quoted, so a package name cannot run in a spreadsheet.

The answer covers only images whose packages Goliash could read (an SBOM or a package database), and the page says
how many those are. While
advisories are still being looked up it says so, and does not answer "not running".

Only package names and versions are sent to OSV, never image, service or host names. `GOLIASH_OSV_URL` points the
lookups at a mirror, or turns them off (`off`).

**Products built into their image.** PostgreSQL, nginx, Redis, Keycloak and most official images build the product
from source, so it is in no package database and often only loosely in an SBOM. Goliash looks up the running
version of the product itself, from the best source there is:

1. **the project's own GitHub security advisories**, when its repository publishes any (Redis, Keycloak, Grafana,
   Traefik, Caddy, Envoy, Argo CD, RabbitMQ, NATS, CoreDNS, Prometheus…): the affected versions and the fix as the
   authors write them, read every 6 hours. Ranges are read as projects write them (`< 26.4.14`, `>= v3.0.0, <=
   v3.7.15`, `All` with a fix per release line); an advisory without a fix and without an upper bound counts for
   half a year after it is published. Then NVD is not asked about that product;
2. **[NVD](https://nvd.nist.gov)** for the others (PostgreSQL, nginx, PHP, Vault…), by the catalog's `cpe` or the
   version policy's, once a day per product version: `postgres:16.2-alpine` is PostgreSQL 16.2, and a moving tag
(`postgres:16`) counts by the version its digest resolved to. Each CVE comes with its severity (NVD's own CVSS
score first) and the first fixed version of its range (*postgresql 16.2, fixed in 16.3*), next to the SBOM's
findings: the same CVE from both is one finding, CISA's exploited list applies to it, and a critical or exploited
one sends the `vulnerability` alert. Without an [API key](/reference/configuration/) NVD answers 5 requests in 30
seconds, so Goliash asks for 8 product versions per check; `GOLIASH_NVD=off` stops it.

## Exposed to the outside

An exploited vulnerability in an internal batch job and the same one behind the shop's ingress are not equally
urgent. Agents report how each workload can be reached from outside its cluster or host: an Ingress or HTTPRoute
host, a LoadBalancer or NodePort service, an ECS load balancer or public IP, a Lambda function URL without
authentication, a Docker port published on every interface (details and permissions in
[Collectors](../../reference/collectors/#exposure)).

- **The service page** lists the ways in under each environment marked **exposed** in Running; its Security panel
  below has the vulnerabilities of those images.
- **Updates** marks such an upgrade **exposed** (click the badge for the ways in) and puts it first within its urgency: an
  exploited vulnerability behind an ingress before one in an internal service, then production first as before.
- **Vulnerabilities** puts findings in exposed images right after those exploited in the wild, marks them, and
  **Only exposed** narrows the list (`?exposed=1`; the CSV and `GET /api/v1/vulnerabilities?exposed=true` too).
- The CSV exports and the API (`exposed` on updates and on each affected image) say how.

What the agent reads is a hint, not a scan: a load balancer may be internal, a firewall may close a port. Label a
workload `goliash.exposed=internal` (or `internet`) where you know better.

## Verify images in CI

Everything above looks at what already runs. `goliash verify` looks before: a step in the pipeline that fails
when the image about to be deployed

- carries a vulnerability CISA lists as **exploited** in the wild,
- is on a release cycle past its **end of life**,
- or is **not signed** as its service's policy wants (`signer` in the version policy): unsigned, or signed by
  someone else.

```sh
GOLIASH_URL=https://goliash.example.com GOLIASH_TOKEN=glsh_api_… goliash verify ghcr.io/acme/payments-api:1.6.0
```

```
ghcr.io/acme/payments-api:1.6.0 (sha256:3f1c…): blocked
  BLOCK exploited: exploited in the wild (CISA KEV): CVE-2021-44228
  warn  critical: 2 critical vulnerabilities: CVE-2024-0001, CVE-2024-0002
  412 packages (SBOM: attestation), signed by https://github.com/acme/payments-api/.github/workflows/release.yml@refs/heads/main; policy of payments-api
```

The exit code is 1 when the image breaks a blocking rule, and 0 when it passes; warnings do not fail it.

- **Where the packages come from.** `-sbom sbom.cdx.json` hands in the SPDX or CycloneDX JSON the build made (syft,
  docker buildx `--sbom`, Trivy). Without it, the server reads the image's SBOM attestation, or its package
  database, from the registry with the workspace's registry credentials.
- **Which rules block.** `-fail-on` chooses: `exploited`, `eol`, `signer`, `unsigned`, `critical`, `no-sbom`,
  `unreadable`, or `none`. The default is `exploited,eol,signer,unsigned`.
- **Policy and output.** `-service` names the service whose policy applies; by default it is the one already
  running the image's repository. `-json` prints the verdict for other tools.
- **Token and audit.** The token can be a *viewer* API token. Each check is in the audit log as `image.check`.
  The API is `POST /api/v1/check`.

GitHub Actions, after the image is pushed:

```yaml
- name: Verify the image with Goliash
  run: |
    docker run --rm -e GOLIASH_URL -e GOLIASH_TOKEN -v "$PWD:/w" ghcr.io/pipozzz/goliash:latest \
      verify -sbom /w/sbom.cdx.json ghcr.io/acme/payments-api:${{ github.sha }}
  env:
    GOLIASH_URL: https://goliash.example.com
    GOLIASH_TOKEN: ${{ secrets.GOLIASH_TOKEN }}
```

GitLab CI, with the release's binary (the server image has no shell to run a script in):

```yaml
verify:
  image: alpine:3.20
  script:
    - V=1.151.0 # your server's version
    - wget -qO- https://github.com/pipozzz/goliash/releases/download/v$V/goliash_${V}_linux_amd64.tar.gz | tar xz goliash
    - ./goliash verify "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"
```

## When the distribution has no fix

*No fix yet* in a distribution's advisory is about that distribution's package in that release. Before Goliash
says it, it looks further: a newer release of the distribution, or the upstream project, may have fixed it.

### Fixed in a newer release of the distribution

A distribution often fixes a vulnerability in its next release before (or instead of) the one an image is built on:
Debian 13 has a fixed nginx for CVE-2023-44487, Debian 12 none. When the image's own release has no fix and a newer
one has, Goliash says so instead of *no fix yet*: *fixed in Debian 13 (trixie)*, and "What to do" says to rebuild on
a trixie base. Only releases to move a production image to count: a stable Debian (not testing, such as forky
before it is released), an Ubuntu LTS. When it knows what the image is built on (its `org.opencontainers.image.base.name` label, its
provenance, or the service's [base image](#base-images)), it names the image to build on instead: *rebuild on
python:3.15-trixie instead of python:3.15-bookworm*; `node:22-alpine3.20` becomes `node:22-alpine3.21`,
`ubuntu:jammy` `ubuntu:noble`. A slim variant also leaves out the build tools and `-dev` libraries a runtime image
does not need, and with them most of a full image's findings. Debian, Ubuntu (not Pro)
and Alpine releases count; the finding counts as fixable, so an exploited one asks for the rebuild, not a
mitigation.

### Upstream fixes

An image often installs the product from upstream rather than from its distribution (the official nginx image has
nginx.org's `nginx 1.27.1-1~bookworm`), and upstream may have fixed a vulnerability long before the distribution.
When a distribution's advisory has no fix for a package whose upstream product Goliash knows (nginx, OpenSSL, curl, PostgreSQL, Redis and others), it asks NVD for the versions
the CVE affects upstream (exploited and serious ones first; `GOLIASH_NVD_API_KEY` makes it faster) and reads the
version that runs without the distribution's revision (`1.27.1-1~bookworm` is nginx 1.27.1):

- **Past upstream's fix:** the package leaves the findings, and the service's page lists it under **Not affected**
  with the ranges and *(NVD)*, as does the monthly report. Where NVD only says the last affected
  version (*up to 1.25.2*) rather than the fixed one, this needs a later release line (1.27 is, 1.25.4 is not):
  NVD may have closed such a range at the newest release of its time, before there was a fix.
- **Still in upstream's range:** instead of *no fix yet* it says which version fixes it (*upstream nginx fixed it
  in 1.25.3*), and "What to do" says to run that version or newer, from upstream's own packages or image when the
  distribution has none. An exploited vulnerability with an upstream fix asks for the upgrade, not a mitigation.
- **Which version to run.** Beside a fix it names the product's newest release (*· latest 1.31.8*) and, when the
  release cycle that runs is still supported, the newest of that cycle too (*on 1.30, 1.30.4*), only when they fix
  it. Releases come from [endoflife.date](https://endoflife.date) for most products, else from the project's GitHub
  releases (curl, zlib), refreshed daily.

## Exploited, no fix yet

Sometimes CISA lists a vulnerability as exploited in the wild before any fixed version exists, so there is no
upgrade to make. (Goliash first looks for one: a newer release of the image's distribution, or an upstream fix,
counts as a fix and asks for the upgrade instead.) Until there is, a **mitigation** keeps it from being used: a WAF or proxy rule, a feature or
setting turned off, the service cut off from the network.

- **Where it shows.** The Security page's **Exploited, no fix yet** panel and the vulnerabilities list (*no fix*)
  show where such a vulnerability runs, and whether each place is mitigated.
- **Recording one.** **Record a mitigation** (members) takes the place (service and environment), what kind it
  is, what was done, and when to review it (2 weeks, 30 or 90 days). It is in the audit log, and the monthly
  report lists mitigations.
- **The deadline.** With remediation deadlines set, the deadline for an exploited vulnerability is the date to
  mitigate by: from when CISA listed it while it ran.
- **Notifications.** The `deadline` notification says, once, where production runs one without a mitigation,
  and three days before a mitigation's review.
- **When a fix comes out**, Goliash closes the mitigation and says so (*a fix for CVE-… is out; upgrade now*). From
  then on the upgrade is on Updates with its deadline, like any other. A mitigation also closes when the
  vulnerable image no longer runs, and a member can withdraw one.

## Not affected

A scanner matches packages against advisories, and sometimes the advisory does not fit what runs. Goliash leaves out
by itself what it can tell does not apply, and lists each under **Not affected** on the service's page and in the
monthly report, grouped by reason, with *Goliash* as who said so:

- a package past the version upstream fixed it in ([Upstream fixes](#upstream-fixes)): nginx.org's `nginx
  1.27.1-1~bookworm` matched against Debian's advisory for CVE-2023-44487, which nginx fixed in 1.25.3;
- what the image's publisher rules out in VEX ([below](#vex-from-the-images-publisher));
- the kernel's packages in a container image ([below](#kernel-packages)).

For the rest, when you know a vulnerability does not affect a service, say so instead of leaving it open:

- **Marking one.** On the service's page, **Not affected?** under the vulnerability (admins) takes the reason (after
  VEX's justifications: already fixed in the version that runs, the component is not in the image, the vulnerable
  code is never reached, the software prevents it another way, or other), a sentence on how you know, and when to
  review it (90 days by default).
- **What it changes.** The vulnerability leaves that service's findings everywhere: the vulnerabilities page,
  alerts, remediation deadlines, Security and the API. Other services that have it still count it. The service's
  page lists it under **Not affected**, with the reason, and the monthly report has a **Not affected** section for
  auditors.
- **Withdraw** (admins) puts it back.

### VEX from the image's publisher

An image can carry an [OpenVEX](https://openvex.dev) document in which its publisher says which vulnerabilities its
packages match but do not affect it: Docker Hardened Images and Chainguard ship them, and your own builds can too:

```sh
# with the build
docker buildx build --attest type=vex,filename=app.vex.json --push -t ghcr.io/acme/app:1.4.0 .
# for an image already pushed
cosign attest --type openvex --predicate app.vex.json ghcr.io/acme/app@sha256:…
```

Goliash reads it beside the SBOM (BuildKit attestations, cosign's `.att` tag, OCI referrers; agents read it for
private registries) and refreshes it daily. A statement that a vulnerability is `not_affected` or `fixed`, for the
whole image or for the package it matched, leaves that package out of the findings and lists it under **Not
affected** with its justification and author (*the vulnerable code is never run, says Acme Security (VEX)*).
Supply-chain evidence shows *vex* among what was found.

### Kernel packages

An image often carries the kernel's packages without running them: `linux-libc-dev` comes with build-essential,
`kernel-headers` with gcc on Red Hat-like images. Every kernel CVE matches them, but a container runs on its host's
kernel, so Goliash leaves them out of the image's findings and lists them, grouped, under **Not affected** (*a kernel
package in the image*). The host's kernel is patched with the nodes; dropping those packages from the final stage of
the build makes the SBOM smaller too.

## Images without an SBOM

Goliash knows an image's vulnerabilities from the packages it contains. It takes them from the first of:

1. **the image's SBOM**: the SPDX or CycloneDX attestation `docker buildx build --sbom=true` attaches (official
   images have one), a cosign attestation, or an SBOM pushed as an OCI referrer. It lists the operating system's
   packages and the application's own dependencies (npm, Composer, Maven, Go modules…);
2. **the image's package database**, when it has no SBOM: Goliash reads the layers, newest first, for
   `/var/lib/dpkg/status` (Debian and Ubuntu; distroless images' `/var/lib/dpkg/status.d`) or
   `/lib/apk/db/installed` (Alpine, Wolfi, Chainguard), and `/etc/os-release`. An image built `FROM ubuntu:22.04`
   with `apt-get install nginx php8.1-fpm` is checked against Ubuntu's own advisories, which know Ubuntu's patched
   builds: `nginx 1.18.0-6ubuntu14.4` is not taken for an old upstream 1.18. The application's own dependencies come
   from their manifests in the same layers: `node_modules/*/package.json` (else `package-lock.json`), Python
   `*.dist-info/METADATA`, `pom.properties` in jars, the build information of Go executables (a `scratch` image
   with one Go binary is no longer blind), gemspecs and .NET `*.deps.json`. Directories the distribution owns
   (`usr/lib`, `usr/share`, `bin` …) are left out: a Debian `python3-urllib3` is known as Debian's package, with its
   patches, not matched again against PyPI's advisories. Files a later layer deleted (`npm prune`, `rm -rf`) do not
   count. Images up to 1 GB of layers (gzip or plain), two per check, are read once
   per digest; private images are read with the workspace's
   [registry credentials](/reference/collectors/#credentials-on-the-server), or by the agents (1.141 and later)
   for registries only they reach (agents from 1.177 read the application's packages too). Images read before
   1.177 are read again once, for their application packages.

**.zip Lambda functions** have no image: their agent reads the packages of their code and layers instead
(npm, PyPI, Maven, Go, RubyGems, NuGet; see [Collectors](/reference/collectors/)), and their vulnerabilities show
under the function like an image's.

What neither covers, Goliash does not know by itself: software built from source or copied into the image, RPM-based
images (Red Hat, Rocky, Amazon Linux), scratch images. **Declare** what you built into the image with a label, and
Goliash looks up its CVEs in NVD by version, like [the products of known images](#is-it-running):

```dockerfile
LABEL dev.goliash.components="nginx=1.27.2, php=8.3.12, openssl=3.3.2"
```

Usual names are known (nginx, php, openssl, node, python, ruby, go, postgresql, redis, haproxy, curl, httpd, tomcat,
openjdk, imagemagick, ffmpeg, git, sqlite, zlib, libxml2, mariadb, mysql, memcached, caddy, traefik, envoy,
keycloak, grafana); anything else by its NVD product, `vendor:product=version`
(`apache:http_server=2.4.62`). A build argument keeps it in step with the build: `ARG NGINX_VERSION` …
`LABEL dev.goliash.components="nginx=${NGINX_VERSION}"`. The label is read with the image's evidence, by the
server or, for registries only they reach, by the agents (1.141 and later). The Security page lists production's images without an SBOM, why
(*no package database*, *not read yet*), and how to add one to the build, ready to copy:

```yaml
# GitHub Actions: the build-push step
- uses: docker/build-push-action@v6
  with:
    push: true
    sbom: true              # an SPDX SBOM of every package in the image, attached to it
    provenance: mode=max    # what it was built from, including its base images
```

```sh
# GitLab CI or anywhere else with BuildKit
docker buildx build --push --sbom=true --provenance=mode=max -t registry.example.com/team/app:1.2.3 .

# an image already pushed, without rebuilding it
syft registry.example.com/team/app:1.2.3 -o spdx-json > sbom.spdx.json
cosign attest --yes --type spdxjson --predicate sbom.spdx.json registry.example.com/team/app:1.2.3
```

At its next deploy (or check), Goliash reads the SBOM from the registry, and it replaces the package list read from
the image.

## Licenses

The same SBOMs say which license each package is under. **Security → Licenses** sorts the packages of running
images by what their licenses ask, most demanding first:

| Class | Examples | What it asks |
| --- | --- | --- |
| Network copyleft | AGPL, SSPL, OSL | offering the software as a service can oblige you to publish your source |
| Source-available | BUSL, Elastic, PolyForm, non-commercial | limits on production, commercial or competing use |
| Strong copyleft | GPL, EUPL | shipping it with your code can oblige you to publish that code |
| Weak copyleft | LGPL, MPL, EPL | changes to the package itself must be shared |
| Unknown | | the SBOM names no license, or one Goliash does not know |
| Permissive | MIT, Apache, BSD | attribution only |

The page opens on the packages that **need a look** (network copyleft, source-available, strong copyleft and unknown),
each with the services and environments it runs in. An expression with OR counts as its least demanding choice
(`MIT OR GPL-2.0` is permissive); one with AND as its most demanding part. Operating system packages (Debian,
Alpine, RPM) are left out unless asked for: copyleft is the norm there (glibc, bash) and comes with the base image.

**Export CSV** on the page downloads what it shows, one row per package and where it runs, for a license review;
`GET /api/v1/licenses?format=csv` gives the same. The same list is `GET /api/v1/licenses` (`?class=attention`, `?environment=prod`, `?os=false`, `?q=AGPL`), the
In a terminal, `goliash licenses` lists the same, and `-fail-on network-copyleft,source-available` exits with 1 when
such a package runs, for a check in cron or CI:

```sh
goliash licenses -environment prod -fail-on network-copyleft,source-available
```

`licenses` tool for AI assistants and Goli, the metric `goliash_license_packages{class,kind}`, and licenses per
package in the [CycloneDX inventory](../sbom/). Goliash reads licenses as the SBOMs declare them: it is a map, not
legal advice.

## Time to patch, and patching by team

Every upgrade deployed to production closes an exposure: from when the first release after the old version came out
(its publication date, else when Goliash found it) to the deploy. Those days are the **time to patch**.
**Time to patch in prod** draws it month by month for the last six months, the median and the 90th percentile, with
how many upgrades each month had; rollbacks and staging do not count, and a version deployed to several targets
counts once.

**Patching by team** sums production up per team (a service's owner): services in production, how many run behind
a newer release and for how long (median), how many upgrades are past their deadline, how many were deployed in the
last 90 days and their median time to patch, with bars for the last six months. The slowest come first: most past
their deadline, then longest behind; each team links to its Updates. The same numbers are
[metrics](/guide/observability/) by owner for a team's dashboard (*Patching by team* in the Grafana and SigNoz
dashboards), the weekly report names the slowest teams, and a team's own rule sends it
[its week](/guide/notifications/#security-posture-report).

## Trend

Goliash records production's posture every hour (the day's last reading stays) and keeps a little over a year of
it. Under the tiles, **Trend** draws the last 90 days of the median exposure, how many services run behind a newer
release, how many for 90+ days and how many past end of life, each with where it started: green when it went down.
The [weekly report](../notifications/#security-posture-report) adds the numbers from a week before, and
`GET /api/v1/posture` returns them once a week as `history`, so [Goli](../goli/) can answer "is patching getting
faster?".

## How exposure is counted

Exposure starts when the first release newer than the running one was **published** (GitHub or GitLab release
dates). When a release's date is unknown, Goliash counts from when it first saw the release and shows the number as
a lower bound (`≥ 12 days`); such rows are left out of the median and the percentile. A service running several
versions in one environment counts with its oldest, the one an upgrade has to reach, and names the targets running
it.

The page shows the last environment (production) by default; *Every environment* lists all of them.

## For audits

- **Monthly report** (`/security/report`, also in the command palette) is one month of vulnerability management,
  made to hand to an auditor for NIS2 (Article 21), ISO 27001 (A.8.8), SOC 2 (CC7.1) or DORA: the remediation
  deadlines in force, production's posture at the month's end, every upgrade deployed with its time to patch and
  whether its security fix met the deadline (the *security fixes in time* share), each team's month, what runs
  exploited or past its deadline as the report is made, the accepted risks with who accepted them and until when,
  and how each number is counted. Arrows move between months; **Export CSV** gives the month's upgrades and **Print
  or save as PDF** the whole report. The month-end posture needs Goliash to have run then: it is read from the daily
  trend samples.
- **Export CSV** gives every exposure with its service, application, environment, targets, running version, the
  version to move to, the exposure date and days, whether the date is known, end of life, accepted risk and the
  release notes link: a ready start for a risk register.
- **Export SBOM** gives a CycloneDX 1.6 bill of materials of everything that runs, with the packages of each
  image's SBOM nested inside: for Dependency-Track, a vendor questionnaire or the software inventory NIS2 and the
  Cyber Resilience Act ask for.
- **Dependency-Track**: Integrations → Dependency-Track gives a script for cron or a scheduled CI job that uploads
  one project per environment (`goliash prod`, version `running`) from
  `/api/v1/inventory?format=cyclonedx&environment=…`. Its API key needs `BOM_UPLOAD` and `PROJECT_CREATION_UPLOAD`;
  the default *Automation* team lacks the second, and the upload answers 401 without it.
- **Print or save as PDF** prints the page with the time it was generated, as evidence of patch management for
  frameworks that ask for it (NIS2, DORA, SOC 2, ISO 27001).

- **Weekly report**: a notification rule with the *security posture report* event sends these numbers and the
  longest exposures every Monday (or day) to Slack, Teams, e-mail or any channel. See
  [Notifications](../notifications/#security-posture-report).
- **Trends and alerts**: `/metrics` serves the posture (`goliash_exposure_median_days`, `goliash_exposure_p90_days`,
  `goliash_exposures`, `goliash_end_of_life`, `goliash_accepted_risk`, `goliash_exposure_days`), the Grafana and SigNoz
  dashboards chart it over time (is patching getting faster?), and the Prometheus rules alert on production past its
  end of life and on 90+ days behind. See [Grafana dashboard](../grafana/) and [SigNoz dashboard](../signoz/).

The same data is available from the REST API (`GET /api/v1/posture`, for GRC tools and risk registers; see the
[API reference](../../reference/api/)), to AI assistants over [MCP](../../reference/mcp/) (`posture`), and to
[Goli](../goli/): **Explain** on every exposure asks what the releases in between fix and what to do first.

---

# SigNoz dashboard
Source: /guide/signoz/

> A ready-made SigNoz dashboard for Goliash: what runs where, drift, delivery speed, security posture and image hygiene, scraped by the OpenTelemetry Collector.

Goliash serves what runs where, drift, delivery speed, security posture and its own health at `/metrics`. The
OpenTelemetry Collector that already sends to SigNoz can scrape it, and a ready-made dashboard shows it next to your
traces, logs and service metrics.

![The Goliash dashboard in SigNoz: services behind upstream, open drifts, the oldest drift and running versions, tables of what runs where and of open drift, drift by kind and services behind upstream per environment as rings, and graphs over time](../../../assets/signoz-dashboard.png)

The quickest way is **Settings → Integrations** in Goliash: it creates a viewer token with one click and shows the
collector settings filled in for your server, next to a download link for the dashboard. The steps below do the same
by hand.

## 1. A token for the collector

`/metrics` needs an API token with the viewer role. Create one under **Users → API tokens**, or:

```sh
goliash token create -name signoz -role viewer
```

Save the `glsh_api_…` value in a file and mount it into the collector container, e.g. as
`/etc/otelcol/goliash-token`.

## 2. Scrape Goliash with the OpenTelemetry Collector

Add a Prometheus receiver to the collector that sends to SigNoz, in its own pipeline:

```yaml
receivers:
  prometheus/goliash:
    config:
      scrape_configs:
        - job_name: goliash
          scrape_interval: 60s
          scheme: https
          authorization:
            credentials_file: /etc/otelcol/goliash-token
          static_configs:
            - targets: [goliash.example.com]

service:
  pipelines:
    metrics/goliash:
      receivers: [prometheus/goliash]
      processors: [batch]
      exporters: [otlp]   # the exporter your SigNoz collector already uses
```

Where this goes depends on how SigNoz runs:

- **Docker (Foundry):** the ingester's configuration, `pours/deployment/ingester/ingester.yaml`. Set it through
  `config.data` in `casting.yaml`, since `foundryctl forge` writes `pours/` again, and add the token file to the
  ingester's volumes. On the older Docker Compose install it is `otel-collector-config.yaml`.
- **Kubernetes (Helm):** `otelCollector.config` in the Helm values, with the token from a secret mounted into the
  collector.
- **SigNoz Cloud:** run a collector of your own with the ingestion key, or add the receiver to the one you already
  run.

The metrics keep their Prometheus names and labels, so the queries in [Metrics and alerts](../observability/#metrics)
work as they are.

## 3. Import the dashboard

Download [`deploy/signoz/goliash-dashboard.json`](https://github.com/pipozzz/goliash/blob/main/deploy/signoz/goliash-dashboard.json),
then in SigNoz open **Dashboards → New dashboard → Import JSON** and upload or paste it. The *environment* and
*service* variables at the top apply to every panel.

- **Versions and drift:** services behind upstream, open drifts and the oldest, what runs where, services running
  more than one version.
- **Delivery (last 30 days):** rollouts stuck or down (and a table of them), deploys, rollbacks, the median and slowest lead time from one environment to the next, deploys
  per service and the lead time table. Lead time needs at least two environments; with only one, its panels stay
  empty and SigNoz says it cannot find `goliash_lead_time_seconds`.
- **Security posture:** how long production has run behind available releases (median and 90th percentile, and
  over time: is patching getting faster?), services behind for 30 and 90 days, past end of life, accepted risk, and
  the longest exposed services, and packages by license class (copyleft, source-available and unknown ones that
  need a look).
- **Vulnerabilities:** exploited ones (CISA KEV), those past CISA's due date, how long the oldest exploited one has
  run, critical ones, and vulnerabilities by severity over time.
- **Image hygiene:** moving tags, tags pushed again, untrusted registries, outdated bases, sizes over budget and
  images without a digest.
- **Images:** what pulling each service's image downloads over time and against its size budget, bases with a newer
  build than the image is built on, images missing a signature, SBOM or provenance, and layer counts.
- **Goliash itself:** agents online and stale, snapshots waiting, failing notifications, the server running the
  background work and the running version.

![The Security posture section of the Goliash dashboard in SigNoz: median exposure, 90th percentile, services behind for 30 and 90 days, past end of life and accepted risk, exposure over time, and the longest exposed services](../../../assets/signoz-security.png)

## 4. Deploys next to your services

SigNoz has no annotations. Put the *Deploys per service* panel next to your service's latency or error panels, so a
spike lines up with the deploy that may have caused it, or alert on new versions (below).

## 5. Alerts

The queries in [Metrics and alerts](../observability/#alerts) work in SigNoz alerts as they are: create an alert
with a PromQL query, for example `sum(goliash_outdated{environment="prod"}) > 0` for production behind upstream, or
`goliash_end_of_life{state="passed"} > 0` for production past end of life.

Alerts about versions themselves (new releases, drift, removed services) are better sent by Goliash directly; see
[Notifications](../notifications/).

Using Grafana instead? See [Grafana dashboard](../grafana/).

---

# Single sign-on
Source: /guide/sso/

> Sign in with Google, Microsoft Entra ID, Okta, Keycloak or Authentik, set up in Settings, with roles from the provider's groups.

Goliash signs people in with any OpenID Connect provider: Google Workspace, Microsoft Entra ID, Okta, Keycloak,
Authentik, JumpCloud, GitLab and others. The whole organization uses one provider. The provider's groups can decide
who may sign in and with which role in each workspace.

An owner sets it up under **Settings → Single sign-on**, with no restart. The server needs a secret key to store the
client secret encrypted; it makes one at its first start (`GOLIASH_SECRET_KEY`, see
[Configuration](/reference/configuration/)).

## 1. Register Goliash with the provider

Create a web application (OAuth 2.0 / OIDC client, *confidential*, authorization code flow) with the redirect URI the
page shows:

```
https://goliash.example.com/auth/oidc/callback
```

That is `GOLIASH_PUBLIC_URL` followed by `/auth/oidc/callback`, so the public URL must be the address people use.

| Provider | Where | Issuer URL |
| --- | --- | --- |
| Google Workspace | Google Cloud console → APIs & Services → Credentials → OAuth client ID → Web application | `https://accounts.google.com` |
| Microsoft Entra ID | Entra admin center → App registrations → New registration → Web; then Certificates & secrets → New client secret | `https://login.microsoftonline.com/<tenant ID>/v2.0` |
| Okta | Admin → Applications → Create App Integration → OIDC → Web Application | `https://<your domain>.okta.com` |
| Keycloak | Clients → Create client → OpenID Connect, client authentication on | `https://<host>/realms/<realm>` |
| Authentik | Applications → Providers → OAuth2/OpenID Provider (confidential) | `https://<host>/application/o/<slug>/` |

Copy the **client ID** and **client secret**.

## 2. Fill in Settings → Single sign-on

- **Issuer URL**: from the table above.
- **Button label**: what the sign-in page says, e.g. *Google*.
- **Client ID** and **client secret**: from step 1. Once saved, the secret is never shown again; leave the field
  empty to keep it.

**Save** first checks that the provider answers (its discovery document), so settings that do not work are refused
and the sign-in in place keeps working. The sign-in page then offers the provider's button.

## 3. Decide who gets in

Two ways, pick one:

- **Groups** (recommended). Add a row per group of the provider: the group, the role, and the workspace.
  - *admin*: an admin of the whole organization;
  - *member* or *viewer*: in every workspace, or in the one chosen.

  At every sign-in the provider's groups decide: someone in none of these groups cannot sign in, and roles follow
  group changes. Owners stay owners whatever their groups.
- **E-mail domains**, without groups: people with an address in these domains get a viewer account in the first
  workspace at their first sign-in. Everyone else needs an invitation.

Example:

| Group | Role | Workspace |
| --- | --- | --- |
| `goliash-admins` | admin | |
| `sre` | member | every workspace |
| `team-payments` | member | payments |
| `everyone` | viewer | every workspace |

### Getting the groups into the token

Goliash reads the groups from the ID token's `groups` claim. Change the claim under **More** when the provider uses
another name.

- **Entra ID**: App registration → Token configuration → Add groups claim (security groups). Entra sends group
  *object IDs*: use those as the group names, or turn on *cloud-only group display names*.
- **Okta**: in the app's Sign On tab, set a groups claim (filter, e.g. *Matches regex* `.*`).
- **Keycloak**: add a *Group Membership* mapper to the client (claim `groups`, full group path off).
- **Authentik** and **Keycloak**: add `groups` under **More → More scopes** when the provider sends groups only then.
- **Google Workspace**: Google sends no groups. Use e-mail domains, or SCIM below.

## 4. Test it

**Test a sign-in** runs the whole sign-in with the provider, but signs no one in and changes no account. It shows
the e-mail the provider confirmed, the groups it sent, and what signing in would give (*member in payments, viewer
in every workspace*) or why it would be refused. Test with an account in each group before you rely on it.

## SCIM provisioning

With SCIM the provider also creates, updates and deactivates accounts before anyone signs in, and pushes group
membership; the group rows above apply to it too. Turn it on with `GOLIASH_SCIM_TOKEN`, see
[Configuration](/reference/configuration/#scim-provisioning).

## Set up by environment variables instead

`GOLIASH_OIDC_ISSUER`, `GOLIASH_OIDC_CLIENT_ID`, `GOLIASH_OIDC_CLIENT_SECRET` and `GOLIASH_GROUP_ROLES` (see
[Configuration](/reference/configuration/#oidc-sign-in)) do the same for deployments kept in Git or Helm. When set,
they take precedence and the settings page shows them read-only.

## Turning it off

**Turn single sign-on off** on the same page. Accounts stay; people sign in with an e-mailed link, a passkey or a
password. Keep one owner who can sign in without the provider, so a provider outage does not lock everyone out.

---

# Teams and workspaces
Source: /guide/teams/

> Sign in with passwords, links or OIDC, roles per workspace, API tokens and sessions, and one workspace per team or client.

## Signing in

- **Passwords.** Everyone may set a password on their account page (the avatar in the top bar → Your account), or an admin
  runs `goliash user password -email you@example.com`. Forgot it? Sign in with a link and set a new one: for 15
  minutes after a link or single sign-on, the current password is not asked for.
- **Magic links.** `goliash login-link -email you@example.com` prints a one-time link. The first person becomes the
  owner. With SMTP configured, people request links by e-mail on the sign-in page. A link opens a page that signs
  in with a button, so a chat's link preview or a mail scanner, which opens links but presses no buttons, does not
  use it up.
- **Invitations.** **Settings → People → Invite** gives an e-mail address a role and shows a sign-in link for it,
  which works once, for 7 days; with SMTP configured Goliash e-mails it too. Signing in with it, someone without a
  password yet lands on their account page and is asked to set one, or add a passkey (not with single sign-on). **Sign-in link** beside a person makes
  a new one (an expired invitation, a lost password).
- **Passkeys.** On the account page, **Add a passkey** saves one in the device (Touch ID, Face ID, Windows Hello, a
  phone or a security key). Then **Sign in with a passkey** on the sign-in page, or pick it from the browser's
  suggestions in the e-mail field: no password, no code. A passkey only works on this server's address, so it cannot
  be phished. Passkeys need `GOLIASH_PUBLIC_URL` with a domain name over HTTPS (or `http://localhost`).
- **Single sign-on (OIDC).** Step by step in [Single sign-on](/guide/sso/). Any OpenID Connect provider (Google, Microsoft Entra ID, Okta, Keycloak, Authentik, …),
  one for the whole organization. An owner sets it up under **Settings → Single sign-on**: the page shows the redirect
  URI to register with the provider (`<public URL>/auth/oidc/callback`), takes the issuer, client ID and secret
  (encrypted with the server's secret key), and maps the provider's groups to roles in a table: a group to admin of
  the organization, or to member or viewer in every workspace or one. With groups, the provider decides at every
  sign-in who gets in and with which role; without, people need an invitation or an allowed e-mail domain. Saving
  checks that the provider answers, and **Test a sign-in** shows what an account would get (its e-mail, its groups,
  the roles) without signing anyone in. The same can be set in the environment instead (`GOLIASH_OIDC_*`,
  `GOLIASH_GROUP_ROLES`, see [Configuration](/reference/configuration/#oidc-sign-in)); it then takes precedence and
  the page shows it read-only.

## Apps with access

AI assistants (Claude, Cursor, VS Code) connect to Goliash's MCP server by signing in as the person who adds them:
a consent page names the app, the person picks a workspace and allows it, and the app reads Goliash as them, read
only. **Your account → Apps with access** lists them, with when each last read, and **Revoke** ends one. An app
loses access when its person leaves the workspace, and after 30 days unused. See [MCP](/reference/mcp/#remote-http).

Admins see every app reading the workspace under **Users → Apps with access**: the app, whose it is, when it was
allowed and last read; **Revoke** ends anyone's. Allowing and revoking are in the audit log. One address can
register at most twenty apps an hour.

## Two-factor sign-in

On their account page, anyone can turn on codes from an authenticator app (1Password, Bitwarden, Google
Authenticator, Authy…): scan the QR code, enter the first code, and keep the ten **recovery codes** shown once. From
then on Goliash asks for a code after the password **and** after a sign-in link, so an e-mailed link alone does not
get in. Single sign-on (OIDC) leaves this to the identity provider.

- A code works once; a little clock drift is fine. Five wrong codes end the attempt for a while.
- Lost the phone: sign in with a recovery code, then set up again or make new codes on the account page.
- With a passkey, **Use your passkey** on the code page works instead of a code.
- No recovery codes either: an admin chooses **Reset 2FA** on **Workspace settings** (it removes passkeys too), or the
  operator runs `goliash user 2fa -email you@example.com -reset`. The `GOLIASH_RECOVERY_EMAIL` link (below) also
  skips the code.

The secret is encrypted with the server's secret key, like channel secrets; recovery codes are stored as hashes.

An owner can **require** two-factor sign-in for the organization (**Workspace settings**): everyone who signs in with a password
or a link then sets up an app, or adds a passkey and signs in with it, before anything else. A passkey counts as
both factors. Owners turn it on only from a session that meets it themselves.

## Locked out?

The images have no shell, but the `goliash` binary in them prints a sign-in link or sets a password. Run it next to
the server, on the same database:

```sh
docker exec goliash goliash login-link -email you@example.com          # Docker: one-time sign-in link
echo 'a long new password' | docker exec -i goliash goliash user password -email you@example.com
kubectl -n goliash exec deploy/goliash -- goliash login-link -email you@example.com
nomad alloc exec -task goliash <alloc-id> goliash login-link -email you@example.com
```

With an empty database (a container recreated without its volume) the first `login-link` creates you as the owner.

**No exec, or a console that wants a shell** (Nomploy, many PaaS): set `GOLIASH_RECOVERY_EMAIL=you@example.com` on
the server and restart it. Every start then logs a one-time sign-in link for you (the log line starts with
`recovery:`); open it within 15 minutes, set a password on your account page and remove the variable again. It
only works for someone who already has an account, or creates you as the owner when nobody has one yet. In the
Nomad pack the variable is `recovery_email`.
Keep `/data` on a volume: it holds the database and `goliash.key`.

## Sessions

A session ends after 30 days (`GOLIASH_SESSION_TTL`) or after 14 days without use (`GOLIASH_SESSION_IDLE`),
whichever comes first.

The account page lists every browser you are signed in on, with its address and when it was last used; sign any of
them out, or all but this one. Changing your password signs out the others. On **Settings → Workspace settings** admins sign a person
out everywhere or remove their password (when it leaked: they sign in with a link and choose a new one).

## Roles

| Role | Can |
| --- | --- |
| viewer | read everything in the workspace |
| member | also map services, edit version policies and acknowledge |
| admin | also manage agents, targets, channels, rules, API tokens and people |
| owner | everything, in every workspace |

Viewer, member and admin are granted **per workspace**. Admin and owner can also be granted for the whole
**organization**: they then reach every workspace.

Organization admins change roles on **Settings → Workspace settings**. Only owners grant or take the owner role, the last owner cannot
be removed, and nobody changes their own role. Every change is recorded in the audit log.

## Workspaces

A workspace has its own agents, targets, services, history, notifications and API tokens. Use one per client (for
an MSP) or per team.

```sh
goliash workspace create -name "Client A" -slug client-a -envs     # -envs adds dev, staging and prod
goliash user create -email ops@client-a.example -role member -workspace client-a
goliash user grant -email ops@client-a.example -role admin -workspace client-a
goliash matrix -workspace client-a
```

Organization owners and admins switch workspaces in the top bar. Everyone else sees only the workspaces they were
invited to, and a workspace admin manages the people of that workspace only, so clients never see each other.

Every CLI command takes `-workspace` (or `GOLIASH_WORKSPACE`); without it, it works on the first workspace.

## API tokens

API tokens (`glsh_api_…`) belong to one workspace. A **viewer** token (the default) reads everything; a **member**
token can also acknowledge. A token may expire after 30, 90 or 365 days, or never. Create them on **Settings → Workspace settings** or
with `goliash token create -name prometheus [-role member] [-expires 90d]`; the token is shown once. **Settings → Workspace settings**
lists every token with who created it and when it was last used, and revokes one at once (`goliash token revoke
-name prometheus`). Things done with a token show its name in the audit log and on acknowledgements.

## Audit log

Every change to configuration, tokens and roles, from the UI, the API or the CLI, and every sign-in is recorded.
Admins see the log on **Settings → Workspace settings**. Secrets never appear in it.

---

# Versions and drift
Source: /guide/versions/

> Version policies, upstream release checks, release notes, end-of-life dates and the kinds of drift Goliash reports.

## Where upstream versions come from

For each service, Goliash picks an **upstream repository**: the one set on the service, else the image its main
containers run. It lists that repository's tags and keeps the newest versions the policy accepts as releases.

- **Public registries** are checked by the server: Docker Hub, GHCR, Quay, registry.k8s.io, gcr.io,
  public.ecr.aws, mcr.microsoft.com, registry.gitlab.com and docker.elastic.co.
- **Private registries**, and private repositories on the public ones, are checked by the server when an admin
  gives it read access under **Settings → Private registries**, else by the agents with credentials from their own
  environment: a configured credential, `docker login`, or Kubernetes pull secrets. See
  [Collectors](/reference/collectors/#private-registries).

New services are checked within a minute, then every hour. **Check upstream now** on a service page, or
`goliash check`, checks at once.

![A service page: its team, upstream and the latest acceptable release; the Argo CD panel with the applications that deploy it, synced or out of sync, with Open PR → 1.7.0; and its versions over time per environment and cluster](../../../assets/service.png)

## Moving tags

Tags such as `1`, `18-alpine` or `latest` move: they point at whatever release was pushed last. The image that runs
has a digest, though, and the server compares it with the digests of the tags it could be (`1.27.3`, `1.27.2`, …,
newest first). The match shows next to the tag, as **1 = 1.27.3**, and from then on drift and upstream releases
are compared with that exact version: `1 = 1.27.3` is behind `28.0.0`, not "1 behind 28".

Each digest is looked up once (a digest never changes); a digest no tag matches is tried again a day later. The
lookups use `HEAD` requests, which Docker Hub does not count against its pull limit, and read full manifests only
for the two newest candidates when the runtime recorded a platform's digest instead of the index's. This needs the
digest the image runs with: Kubernetes, Docker, Swarm and Nomad report it; Compose files from Git do not. For
images on private registries the agents do the lookup with their credentials, after their next registry check.

## Badges

A service's page offers badges for a README, a wiki or a status page: the running version and its state, in the
state's colour, for one environment or for all of them.

![Four badges: checkout in staging 2.4.0 up to date in green, checkout in prod 2.3.1 behind in amber, keycloak in prod 24.0.5 needing attention in red, and keycloak's version in every environment](../../../assets/badges.png)

**Copy markdown** next to a badge gives a link that opens the service in Goliash:

```markdown
[![checkout in prod](https://goliash.example.com/badge/…/checkout.svg?env=prod&sig=…)](https://goliash.example.com/services/checkout)
```

Badges load without signing in, so GitHub and wikis can show them; they refresh within five minutes. Each address is
signed for its service and environment, so nobody can make one up for another service. Members and up see the badge
addresses. `goliash badges reset` makes a new signing key: every badge handed out stops loading, and the service pages
offer new ones.

## Own or third-party

A service is *third-party* when its image is someone else's software: one of Docker Hub's official images
(`docker.io/library/postgres`) or an image of the [catalog](#version-policy); anything else is taken as your team's
*own*. The service page shows it under **Kind**; its form can set it either way, or back to *from the image*.

## Version policy

A policy decides which tags count as versions and which differences are worth an alert. Set it on the service
page, with `goliash service set`, or through the catalog.

| Field | Meaning | Default |
| --- | --- | --- |
| `track` | The smallest jump worth an alert: `patch`, `minor` or `major`. | `patch` |
| `pin_major` | Stay on this major: newer majors are shown, not alerted. | none |
| `tag_filter` | A regular expression tags must match. | tags shaped like the running one |
| `prerelease` | Consider alpha, beta and rc versions. | `false` |
| `auto_pr` | Per environment, the biggest jump Goliash opens a pull request for by itself, in the repository its [Argo CD or Flux](/guide/argocd/#automatic-pull-requests) application deploys from: `{"dev": "minor"}`. | none |
| `drift_alert_after` | How long drift lasts before it is announced, per kind, e.g. `{"env": "72h"}`. | see below |
| `github`, `github_tag_prefix` | GitHub repository (`owner/repo`) whose releases give dates and release notes. | from catalog or image label |
| `gitlab` | GitLab project (`gitlab.com/group/project`) whose releases give dates and release notes; `github_tag_prefix` applies too. | from catalog or image label |
| `changelog` | A release notes URL template with `{version}`. | none |
| `cpe` | The product in [NVD](https://nvd.nist.gov), as CPE `vendor:product` (a list): its CVEs are looked up by the running version, for software built into the image that no SBOM or package database lists. See [Is it running?](/guide/security/#is-it-running) | from catalog (`postgresql:postgresql` for `postgres`) |
| `security` | Where the project says which releases fix vulnerabilities, when its release notes do not: `php.net`, `go.dev`, or `eol-phase` (every release of a cycle in security-fixes-only support). Releases whose GitHub or GitLab notes name a CVE, GHSA or GO id (or say *security release*) are known without it. See [Security releases](/guide/notifications/#security-releases). | from catalog (`php`) |
| `base_image` | The image the service's image is built `FROM`, when it does not declare it; `none` turns base tracking off. | as the image declares |
| `max_size_mb` | The image size budget in MB: over it, Image hygiene lists it and `image_over_budget` alerts. | none |
| `signer` | Who should sign its images: a keyless signature's identity (`https://github.com/acme/api/.github/workflows/release.yml@refs/heads/main`), or a prefix ending in `*` (`https://github.com/acme/*`). Otherwise Image hygiene lists it as *unexpected signer*. | none |

Without a `tag_filter`, tags are compared **like with like**: a service running `1.27.2-alpine` is compared with
other `-alpine` tags with three version parts, so `latest`, `mainline` or `1.27-bookworm` do not count. A tag that is no
version names a flavour: `alpine` is compared with the newest `8.2.1-alpine`, `pg18` with `0.8.1-pg18`, while
`latest` or `stable` are compared with plain releases. Tags such as `-testing`, `-unstable` or `-insiders` count as
prereleases.

```sh
goliash service set -name postgres -track minor -pin-major 15
goliash service set -name app -tag-filter '^v\d+\.\d+\.\d+$'
```

## The catalog

The built-in [catalog](https://github.com/pipozzz/goliash/blob/main/catalog/images.yaml) has default policies for
popular public images (postgres, redis, nginx, keycloak, traefik, grafana, prometheus and more) and where their
release notes live. A service without its own policy uses the catalog's; the service page then says "from catalog".
Additions are welcome as pull requests.

## Release notes

Releases get a publication date and a release notes link from GitHub or GitLab releases. The repository comes
from, in order:

1. `github` or `gitlab` in the service's own policy,
2. the catalog,
3. the image's own `org.opencontainers.image.source` label, read from the registry once a week. Most images built
   with GitHub Actions set it, so they get release notes without any setup. Docker Official Images are skipped,
   because their label points at the repository that packages them, not the project.

The service page says which one is used. Set `GOLIASH_GITHUB_TOKEN` to raise GitHub's rate limit when you track
many services.

GitLab releases are read from gitlab.com and from one self-hosted instance set in `GOLIASH_GITLAB_URL`, with an
optional `GOLIASH_GITLAB_TOKEN` for private projects. Other hosts are never called, because a policy or an image
label must not make the server reach arbitrary addresses.

## Drift

| Kind | Meaning | Announced after |
| --- | --- | --- |
| `env` | An environment runs an older version than the environment before it. | 7 days |
| `upstream` | The version is behind the newest release by at least the tracked jump. | immediately |
| `inconsistent` | Targets of one environment run different versions. | 15 minutes |
| `declared` | What runs differs from what the Compose files in Git declare. | 30 minutes |
| `eol` | The running release cycle reaches its end of life within 60 days, or already has. | immediately |

Where an environment runs more than one version, `upstream` and `eol` are about the oldest of them: a target left
behind while the others moved on still shows up on the Updates page. `env` compares the version most replicas run.

Drift shows in the UI as soon as it exists. It is **announced** as a `drift_detected` event, which is what
notification rules react to, only once it has lasted the time above. That way a normal promotion from staging to
prod over a few days, or a rollout in progress, does not page anyone. Override per service:

```json
{"drift_alert_after": {"env": "72h", "inconsistent": "1h"}}
```

### End of life

Goliash reads support dates from [endoflife.date](https://endoflife.date): it knows which product an image is
(postgres → PostgreSQL, through the site's package URLs, or the catalog's `eol` field), picks the release cycle of
the running version (15.6 → 15, 7.2.4 → 7.2) and opens `eol` drift 60 days before that cycle stops being supported,
or at once when it already has. Set `"eol": "postgresql"` in a service's policy to name the product yourself, or
`"eol": "none"` to turn it off. Without internet access, start the server with `-eol=false`.

### Declared versus running

A `compose` target describes what **should** run; `docker`, `kubernetes` and the other targets report what **does**
run. When an environment has both, the matrix shows what runs, and `declared` drift appears when it differs from
the files: "Git says 1.5.0" while prod still runs 1.4.2 because a deploy failed or never happened, or someone
changed a container by hand. When an environment only has Compose files, the matrix shows the declared versions,
marked *declared*.

When the drift ends, a `drift_resolved` event follows.

## Updates

**Updates** is a to-do list of upgrades: one line per service (and application) and environment with open end of
life, upstream or environment drift. Each line shows what runs, the version to move to, with its release notes, and
why. They are ordered by urgency (an exploited vulnerability in the running image, a [security
fix](../notifications/#security-releases) the project announced for a release in between, end of life passed, end of
life soon, major, minor, behind the previous environment, patch), production first, then by how long they have been
open. A *security fix* line says what it fixes (*fixes CVE-2026-…* or *security fix*), and a service under a
[security watch](../security/#security-watches) carries *watch: php*.
When the release notes on the way (GitHub or GitLab releases, or the project's own notes pages) say something breaks,
the line carries *breaking in 2.0.0* (or *in 3 releases*) and quotes them: *2.0.0: Removed the v1 API*. Goliash looks
for breaking-change headings and labels, `feat!:` commits, removed support, required migrations and incompatible
changes; read the notes themselves before upgrading, as a project may say it otherwise or not at all.
With [remediation deadlines](../security/#remediation-deadlines) set, a line says *due in 3 days* or *overdue 5 days*,
and a *past deadline* tile shows only those.
Where the running image's [SBOM](../sbom/) is known, a line names the vulnerabilities CISA lists as exploited in the
wild (they put it first) and counts the other critical ones, each a link to [Is it running?](../security/#is-it-running).
When the version to move to already runs somewhere (staging, another cluster) and both images have an SBOM, the line
also says what the upgrade does to known vulnerabilities: *fixes 8 vulnerabilities (1 exploited) · brings 2 new ones*,
and the weekly upgrade plan says what each upgrade fixes. The tiles filter by urgency, and the
selects by team and environment; **+ Save view** keeps a team's or an environment's list as a tab (see
[Saved views](/guide/matrix/#saved-views)). **Copy as checklist** copies the list as markdown checkboxes for a ticket or a
planning board. **Put off** on a line skips that version (until a newer release is out) or puts it off for a week or
30 days; it is an [acknowledgement](#acknowledging) for that service and environment, so its notifications go quiet
too. Updates put off are kept apart under *Acknowledged*.

Where a GitOps application deploys the service, a line offers **Open PR** to the version to move to; tick several
and **Open the ticked pull requests** opens them together, each on its own or, with *one per repository*, one per
repository, branch and environment (see [Upgrade with a pull request](/guide/argocd/#upgrade-with-a-pull-request)).

**An upgrade campaign** (every nginx to its newest release, say): filter the list (`nginx`, or *Their upgrades* from
[Search](/guide/matrix/)), **Tick all shown**, and open them together; the button counts what goes, and upgrades the
filter hides are left out. **Copy as checklist** copies the upgrades shown, for a ticket.
**Download CSV** gives the same upgrades as a spreadsheet (service, environment, running, target, urgency, end of
life, exploited CVEs, security fixes, breaking changes, watches, deadline, release notes), the filter's words included.

The same list is `GET /api/v1/updates` and the MCP tool `updates`, for an assistant to plan with. A notification rule
with the upgrade plan sends it every week; see [Upgrade plan](/guide/notifications/#upgrade-plan).

![The Updates page: tiles counting all, major, minor and patch upgrades, and a list of services with the running version, the version to move to, why, how long it has been open, Open PR where Argo CD deploys it, and Put off](../../../assets/updates.png)

## Promotions and delivery

The **Delivery** page opens with two charts: versions that arrived per week and environment over the last twelve
weeks, and how much drift was open each day of the last thirty, by kind. Every service page shows **Versions over
time**: which version ran in each environment (one lane per target where an environment has several) over the last
thirty days; hover a bar for its dates. The charts are drawn on the server, follow the light or dark theme and need
no script.

![The Delivery page: deploys per week as stacked bars by environment, and open drift per day as lines over gradient areas](../../../assets/delivery.png)

The **Delivery** page lists every version that runs in one environment and waits for the next, longest waiting
first, with the releases a promotion would bring and their release notes. When both versions' images have an SBOM,
it also says what the promotion does to known vulnerabilities: *fixes 3 vulnerabilities (1 exploited) · brings 1
new one*, with their IDs one click away. It is the list to go through before a release to production;
When the promoted version brings packages under network copyleft, source-available or strong copyleft licenses, the
line says that too (*⚖️ brings AGPL-3.0-only (itextpdf)*). With a GitOps application in the environment behind,
**Open PR → prod** opens the promotion as a pull request; when several services of one application wait together,
**Promote webshop-prod → prod** opens one pull request for all of them. Below, **Pull requests** lists those Goliash
opened in the last 30 days, open, merged or closed, with their checks, whether their version runs and how long after
opening it began to, and the median of that: your time from proposal to production.
`goliash promotions` and `GET /api/v1/promotions` (`vulnerabilities.fixed` and `.new`, `licenses`) show the same.

```sh
goliash promotions
```

**Rollouts that do not finish** come first on the page. Kubernetes, ECS, Nomad, Swarm and Compose say how many
replicas a workload asks for, and Goliash compares that with what runs:

- **down**: none of the replicas runs;
- **stuck**: two versions side by side for more than 30 minutes (*traefik v3.1.6 on 1, v3.1.2 on 1 of 2 replicas
  asked for*), a rolling update that stopped half way;
- **short**: fewer replicas than asked for.

Jobs that are meant to finish are left out: Nomad batch, periodic and parameterized jobs, Nomad system jobs (one per
node, whatever their count says), and Kubernetes CronJobs.

The matrix marks the cell too: a red *rollout stuck* or *down* badge (its tile needs attention), or *2 of 3 replicas*
when it is short. The service page shows the same in red, `GET /api/v1/rollouts` and the `rollouts` tool for AI assistants list them,
and a notification rule with the `rollout` event announces stuck rollouts and services down (not short ones: a pod
restarting is not news).

```text
SERVICE        PROMOTE           FROM → TO        WAITING   VULNERABILITIES   RELEASES
payments-api   1.5.0 → 1.6.0     staging → prod   3d        2 fixed, 0 new    1.6.0, 1.5.1
postgres       15.6 → 15.7       staging → prod   26h       1 fixed, 0 new    15.7
```

Below it, the page shows how versions moved over the last 30 days, read from what ran when rather than from CI:
how many versions arrived in each environment, and the median time a version took from one environment to the
next (its *lead time*, e.g. staging → prod). A change back to an older version counts as a **rollback**, not a
deploy: rollbacks against deploys are DORA's change failure rate, counting a rollback as a failed change. The
history marks them *rolled back*, and `GET /api/v1/delivery` and the metric `goliash_rollbacks` have them.

```sh
goliash delivery -window 720h
```

The same numbers are in `GET /api/v1/delivery?window=30d` and in `/metrics` as `goliash_deploys` and
`goliash_lead_time_seconds`, for dashboards and alerts such as "nothing reached prod for two weeks".

## Monthly report

**Delivery → Monthly report** shows one month of a workspace on one page: services and targets, versions deployed
per environment with lead times, new upstream releases, and everything that needs attention now (end of life,
differences from Git, lagging versions), most important first. With [pull requests](/guide/argocd/) on, it also
counts those Goliash opened that month: merged, still open, closed, by version policies, rollbacks, and the median
time from pull request to running. *Print or save as PDF* gives a clean document,
for a monthly review or, with a workspace per client, an MSP's report to each client. Pick the month with
`/report?month=2026-09`.

## Acknowledging

An acknowledgement silences notifications for a service, optionally in one environment, until a version ships or a
time passes:

```sh
goliash ack -service postgres -kind release -until-version 17.0   # "we know about 16, quiet until 17"
goliash ack -service web -kind drift -env prod -for 336h         # quiet for 14 days
```

The same works on the service page and through `POST /api/v1/acks`.

---

# Widgets
Source: /guide/widgets/

> Goliash's numbers on an iPhone or iPad home and lock screen with Scriptable, in the Mac menu bar, and on the Windows 11 Widgets board: drift, updates, exploited vulnerabilities, how long production runs behind and end of life.

The numbers that matter at a glance, without opening Goliash: services drifting, updates waiting, how many of them
run an image with a vulnerability exploited in the wild, the median days production runs behind a newer release,
how many for 90+ days, and end of life. A tap or click opens the Security page.

![The Goliash widget: 6 drifting, 29 updates, 1 exploited, 43 days median behind, 4 behind 90+ days, 0 end of life](../../../assets/widget.png)

## iPhone and iPad

Apple only lets installed apps put widgets on the home screen, so the widget runs in
[Scriptable](https://scriptable.app), a free app that makes widgets from a short script.

1. **Integrations → Widgets → Create a viewer token** (admins). The token can only read; revoke it any time under
   Workspace settings.
2. On the iPhone, open that page and tap **Download Goliash.js**: the server's address and the token are already
   in it. Open the file in Scriptable (**Share → Scriptable**); it is added as *Goliash*. (On a computer, **Copy**
   and paste it into a new script in Scriptable instead.)
3. Long-press the home screen, **+**, *Scriptable*, pick a size and add it; then tap the widget and choose
   *Goliash*.

**What it runs** under the button shows the script: it reads `/api/v1/summary` with the token and draws the
numbers, nothing else.

Every size works: small shows the median exposure, drift and end of life; medium adds updates, exploited
vulnerabilities and 90+ days; large adds services, how many run behind and stale targets; on the lock screen it is
one line. iOS refreshes widgets about every 15 minutes, as it sees fit. The phone must reach Goliash's address: over
the internet, or through a VPN to an internal one.

## Mac menu bar

macOS keeps Notification Center widgets for installed apps too, so on a Mac Goliash sits in the menu bar instead,
through [SwiftBar](https://swiftbar.app) or [xbar](https://xbarapp.com) (both free):

1. With the token from above, **Integrations → Widgets → Download goliash.15m.sh** (server and token filled in).
2. Move it into the plugin folder SwiftBar or xbar asked for on first start, and run `chmod +x goliash.15m.sh`.

The menu bar shows *Goliash* with a count: red for exploited vulnerabilities or end of life, amber for services
drifting, a ✓ when all is quiet. Its menu lists the numbers, each opening the page behind it, and refreshes every
15 minutes (the `15m` in the name). It needs only `curl` and `plutil`, which every Mac has.

## Windows 11

Installed from Microsoft Edge, Goliash brings its own widget to the Windows 11 Widgets board:

1. Open Goliash in Edge, choose **App available → Install** in the address bar (or *Apps → Install Goliash* in the
   menu), and sign in in the app's window.
2. Open the Widgets board (**Win+W**), **+ Add widgets**, and add **Goliash**.

It uses the app's sign-in, so it needs no token; when that sign-in ends, the widget asks you to open Goliash and
sign in again. It refreshes every 15 minutes and whenever the board opens. Windows shows widgets of installed web
apps only from Edge, and the Widgets board's own settings must allow them.

## For anything else

Both widgets read `GET /api/v1/summary` (see the [API reference](/reference/api/)), with a viewer token or a
session:

```sh
curl -s -H "Authorization: Bearer $GOLIASH_TOKEN" https://goliash.example.com/api/v1/summary
```

```json
{"workspace":"Default","production":"prod","services":18,"drifting":6,"updates":29,"exploited":1,"behind":14,
 "median_exposure_days":43,"behind_90_days":4,"end_of_life_passed":0,"stale_targets":0,
 "generated_at":"2026-10-14T13:05:00Z","generated":"Oct 14, 13:05 UTC"}
```

The same call feeds a wall screen, a status bar (SwiftBar, xbar, Waybar), a Home Assistant sensor or a Stream
Deck key.

---

# What runs where, on which version
Source: /

> Self-hosted, open-source version tracking for Kubernetes, Amazon ECS, AWS Lambda, Nomad, Docker Swarm and Compose. One matrix of services × environments, with upstream releases, drift, end-of-life, vulnerabilities, a CycloneDX SBOM of what actually runs, and Argo CD and Flux next to it, upgrades opened as pull requests.

<input type="radio" name="tour" id="tour-tiles" checked />
<input type="radio" name="tour" id="tour-matrix" />
<input type="radio" name="tour" id="tour-gitops" />
<input type="radio" name="tour" id="tour-promote" />
<input type="radio" name="tour" id="tour-rollback" />
<input type="radio" name="tour" id="tour-security" />

<label for="tour-tiles">Tiles</label>
<label for="tour-matrix">Matrix</label>
<label for="tour-gitops">GitOps</label>
<label for="tour-promote">Promote</label>
<label for="tour-rollback">Rollback</label>
<label for="tour-security">Security</label>

<figure class="tour-tiles"><figcaption>Every application as a tile, coloured by how current it is: by application, team, target or status, on any screen or a TV.</figcaption></figure>
<figure class="tour-matrix"><figcaption>The matrix: every service in every environment, what drifts, and the newest upstream release.</figcaption></figure>
<figure class="tour-gitops"><figcaption>Every Argo CD application and Flux Kustomization on every cluster: what Git pins, what runs, the pull request that moves it.</figcaption></figure>
<figure class="tour-promote"><figcaption>A whole application to production in one pull request: every service's tag, with why and since when it has run in staging.</figcaption></figure>
<figure class="tour-rollback"><figcaption>Degraded right after a deploy? Goliash names the change and offers the rollback, as a pull request.</figcaption></figure>
<figure class="tour-security"><figcaption>How long production has run behind a fix, what is past its end of life, and the pull request that closes it.</figcaption></figure>

Goliash watches what actually runs and tells you when a new release ships or when environments drift apart:
staging on 1.5.0 while prod is still on 1.3.2, or two prod clusters disagreeing.

  
    Kubernetes, Amazon ECS, AWS Lambda, Nomad, Docker Swarm, Docker hosts and plain Compose files side by side, per
    environment, with replicas and rollouts in progress. Grouped by application, team, cluster or status.
  
  
    Workloads are grouped by the application they belong to, from Kubernetes, Helm and Compose labels or Nomad jobs.
    Projects deployed with Dokploy or Nomploy come together under one name. A database image shared by several apps
    is compared within each app, so different apps on different versions are not drift.
  
  
    Tags from Docker Hub, GHCR, Quay, ECR, Google Artifact Registry, Azure ACR and private registries, compared
    with semver policies. Moving tags resolved by digest: `1` is shown as `1 = 1.27.3`. Release dates and notes from
    GitHub, found through a built-in catalog or the image's own source label.
  
  
    The Updates page lists every upgrade with its target version, most urgent first: an image with a vulnerability
    exploited in the wild, a release that fixes vulnerabilities, end of life, then major, minor and patch, production
    first. Filter it, tick all shown and open the pull requests together; copy it as a checklist, or get it every
    Monday as an upgrade plan in Slack or e-mail.
  
  
    Upgrades, promotions and rollbacks open as pull requests on GitHub, GitLab, Gitea or Forgejo, changing the tag in
    manifests, Kustomize, Helm values, Nomad jobs, Compose files or Terraform. Previewed first, with why, the
    vulnerabilities they fix and the licenses they bring; a whole application in one; Renovate's linked, not duplicated;
    held back during a change freeze or an Argo CD sync window.
  
  
    A merged pull request whose version does not run yet is shown and announced, with why it waits: a sync window, a
    suspended Flux object, an Argo CD application synced by hand. The Delivery page measures merge to running.
  
  
    Projects fix before their CVEs reach scanners. Goliash reads release notes and feeds (php.net) and announces a
    security release to every service on an older version the hour it ships. After an advance notice, watch `php`:
    see what runs it now, hear of its next release first.
  
  
    ⌘K jumps to a service by the image, version, workload or GitOps application it runs, to a team, or to where a CVE
    runs. Every page filters with `/`, and one search looks through services, vulnerabilities, packages, pull
    requests and history, for you, your scripts and your AI assistant.
  
  
    An environment behind the one before it, a version behind upstream, a release cycle at its end of life, a
    host that runs something other than Git declares: shown at once, announced only when it lasts.
  
  
    Rollouts stuck half way, services running none of their replicas and rollbacks to an older version, from
    Kubernetes, ECS, Nomad, Swarm and Compose: first on the Delivery page, announced, and counted as a change failure
    rate. Each version waiting for production says which known vulnerabilities promoting it fixes and brings.
  
  
    What pulling each image costs over time, the layers the last deploy added with the Dockerfile step of each, a
    size budget per service, and images built on an outdated build of their base image.
  
  
    Every deploy, rollout, retag and removal, read from the runtime itself, not from pipelines. Scrub the timeline
    back through 400 days and watch versions change, or play the deployments back one by one.
  
  
    Not one image at build time: every image in every environment, read from the runtime, with its digest, where it
    runs and the packages inside, as CycloneDX 1.6. For now or any past moment, straight into Dependency-Track.
  
  
    Slack, Microsoft Teams, Google Chat, Discord, Telegram, ntfy, signed webhooks, Grafana annotations, e-mail and browser push, instant or as daily or weekly
    digests, with acknowledgements and a scheduled upgrade plan per team.
  
  
    How long production has run behind an available release, what is past its end of life, accepted risk and blind
    spots (with why a newest release is unknown, and the fix), read from the runtime. Known vulnerabilities with how long each has run and CISA's due dates, the version or base image that fixes each, and false positives (kernel packages, upstream fixes, the publisher's VEX) left out with their reason. Export it as
    CSV or PDF for the risk register and the audit.
  
  
    The license of every package in running images, AGPL, source-available and unknown ones first, and who signed
    each image, as its Sigstore certificate names the CI workflow or person, with an expected signer per service.
  
  
    Ready-made Grafana and SigNoz dashboards, deploy markers on your graphs and Prometheus alerts. Settings →
    Integrations fills in the scrape settings and the token; the Helm chart adds a ServiceMonitor too.
  
  
    An MCP server built in: Claude, Cursor and others answer "what runs in prod?" or "what changed in the last two
    hours?" from live data. They sign in with OAuth as you, read only; no token to copy. Try it on the demo:
    `claude mcp add --transport http goliash https://demo.goliash.dev/mcp`.
  
  
    Every service has a team, and each team a scorecard: exploited vulnerabilities, end of life, stuck rollouts,
    what is behind, with a brief from Goli. Show one team alone as its own board or TV screen, and save any view as a
    tab or your home page.
  
  
    Pick the platform and copy one command: the agent registers itself and finds what to watch, from a Docker host to
    every ECS cluster and Lambda function of a region. One code serves a whole fleet.
  
  
    Collectors only read; the agent only connects out. One binary each, SQLite or PostgreSQL, signed releases,
    passkeys and two-factor sign-in.
  

## See it

**Updates** turns drift into a to-do list: what runs, what to move to (with release notes) and why, the most
urgent first.

**Security** says how long production has run behind an available fix, what is past its end of life and where
Goliash cannot see, ready to export for the risk register or the audit.

**Is it running?** answers "is Log4Shell in production?" from the images' SBOMs: CVEs exploited in the wild first,
with the service, the package and the version that fixes it.

**Time travel**: scrub back through 400 days, step from deployment to deployment or press Play, and watch the
matrix or the tiles change, coloured as things stood then. "What ran in prod when the incident started?" in seconds.

**Argo CD and Flux**: each service shows the Git repository, path and commit that deploy it, whether it is in sync
and healthy, and what actually runs next to it. An upgrade or a promotion is one click away as a **pull request** on
GitHub or GitLab, changing the tag where your manifests, Kustomize or Helm values pin it; you review, Argo CD or Flux
deploys. Out of sync for an hour? Slack knows. [How it works](/guide/argocd/)

Before anything is sent, the **preview** shows the change line by line and the description, with the
vulnerabilities the upgrade fixes. On [the demo](https://demo.goliash.dev/services/checkout) you can click it.

## Meet Goli

**Goli** is the assistant inside Goliash, the gentle giant with the release bell. Ask it in plain words and it
looks things up for you, live, as you:

What should we upgrade first, and why?
<strong>payments-api in prod</strong>: it runs log4j-core 2.14.1, exploited in the wild; 2.16.0 fixes it.
Then <strong>postgres 15</strong>, end of life in a month…

- **Answers from live data**, with links to the pages that show the details: what runs where, drift, updates,
  vulnerabilities, what changed, any moment in the past.
- **Briefs you on an upgrade**: it reads the release notes between what runs and the target and tells you what
  breaks, what it fixes and what to check.
- **Only reads, as you**: it sees what you may see and cannot change anything.
- **Opt-in and yours**: off until you give your server a model: Claude (the default), OpenAI, or one on your own
  hardware with Ollama or vLLM, so nothing leaves your network.

[Ask Goli →](/guide/goli/)

### What will this upgrade break?

Every upgrade on the Updates page has a **Brief** button. Goli reads the release notes of **every release between what
runs and the target**, straight from the project's GitHub or GitLab releases, and turns them into a checklist:
breaking changes and migrations first, then what it fixes (security first), then what to check before and after.
Keycloak 24 → 26 is three major versions of release notes; Goli reads them in half a minute and starts with what will
stop your login page from working.

<video class="screenshot brief-video" data-autoplay muted loop playsinline controls preload="none" poster="/video/goli-brief-poster.webp" width="1280" height="720" aria-label="Goli briefing an upgrade on the live demo: Brief on keycloak 24.0.5 to 26.0.0 in prod opens Goli's panel, it looks up the service and the release notes, and lists what breaks: hostname v1 and the proxy option removed, the JavaScript adapter no longer served, GELF logging removed, sessions persisted to the database, login theme changes">
  <source src="/video/goli-brief.mp4" type="video/mp4" />
</video>

It never calls an upgrade safe, and it says so when the notes are silent: the decision stays yours. Try it on the
[live demo](https://demo.goliash.dev/updates).

### An SBOM of what actually runs

Build-time SBOMs say what an image contains. Goliash says **which of those images run, where, and since when**: one
CycloneDX 1.6 document with every running image, its digest, the environment, cluster and service it runs in, and the
packages of its SBOM nested inside. Export it from the matrix or the Security page, for one environment or a moment in
the past, or let Dependency-Track track one project per environment:

```sh
curl -H "Authorization: Bearer $GOLIASH_TOKEN" \
  "https://goliash.example.com/api/v1/inventory?format=cyclonedx&environment=prod" \
  | curl -X POST "$DTRACK/api/v1/bom" -H "X-Api-Key: $DTRACK_API_KEY" \
      -F autoCreate=true -F projectName="goliash prod" -F projectVersion=running -F bom=@-
```

The software inventory the Cyber Resilience Act, NIS2 and your customers' questionnaires ask for, kept current by the
runtime itself. [How it works →](/guide/sbom/)

**Tiles** draw your estate like the logo: every application a cell, warm where something needs attention, one board
per environment side by side. Open a cell for its services; put it on a wall screen in TV mode.

**Every target at a glance**: the share of workloads that are current, deploys over time, and every workload as a
card, grouped by application.

**Delivery**: deploys per week, open drift per day, and what waits to be promoted.

## How it works

```
 your network                                    Goliash server
┌─────────────────────────────────┐             ┌──────────────────────────────────┐
│ Kubernetes / ECS / Nomad /      │             │ ingest → mapping → versions      │
│ Swarm / Docker ─read-only─► agent ─HTTPS────► │   │                  │           │
│ private registries ──► agent    │  snapshots  │   ▼                  ▼           │
└─────────────────────────────────┘             │ events, matrix   notifier ──► Slack, webhook, e-mail
                                                │ UI, REST API     SQLite / Postgres│
                                                └──────────────────────────────────┘
```

The **agent** runs next to what it watches and sends full snapshots, so an outage loses nothing. The **server**
turns snapshots into history, maps containers to services, compares them with upstream releases and notifies
you. It can also collect what it reaches itself, without an agent.

## Start here

---

# Install
Source: /install/

> Pick how to run the Goliash server and agents: Kubernetes, Docker, Swarm, Nomad or ECS, with SQLite or PostgreSQL.

Goliash is two programs:

| | What it does | Where it runs | License |
| --- | --- | --- | --- |
| `goliash` | The server: UI, API, ingestion, version checks, notifications | Anywhere your team can reach | AGPL-3.0-only |
| `goliash-agent` | Reads orchestrators and private registries, sends snapshots | Next to what it watches | Apache-2.0 |

An agent is optional: the server can collect targets it can reach itself.

## Images and binaries

- **Images:** `ghcr.io/pipozzz/goliash` and `ghcr.io/pipozzz/goliash-agent`, for linux/amd64 and linux/arm64.
  They are distroless and run as a non-root user. Tags follow releases (`0.1.0`), and `latest` is the newest
  release. The commands on the **Connect** page pin the agent image, chart and files to the server's own release, so
  an agent you add later matches the server instead of whatever `latest` is that day.
- **Binaries:** archives for Linux, macOS and Windows on the
  [releases page](https://github.com/pipozzz/goliash/releases).
- **From source:** `make build`, or `docker build --target server .` and `docker build --target agent .`.

Images and release archives are signed with cosign (keyless, from the release workflow) and come with SPDX SBOMs.
See [Security](/security/#verify-a-release) for how to verify them.

## Pick a platform

| Platform | Server | Agent |
| --- | --- | --- |
| Kubernetes | [Helm chart](/install/kubernetes/) | [Helm chart](/install/kubernetes/#agent) with a read-only ClusterRole |
| Docker and Compose | [Compose quickstart](/install/docker/) | [Compose file](/install/docker/#agent-on-another-host) with docker-socket-proxy |
| Docker Swarm | [Stack file](/install/swarm/) | [Stack file](/install/swarm/#agent) with docker-socket-proxy |
| Nomad | [Job](/install/nomad/) | [Job](/install/nomad/#agent) with a list-jobs and read-job ACL token |
| Amazon ECS | any of the above, with PostgreSQL | [CloudFormation template or Terraform module](/install/ecs/) with a read-only task role |
| AWS Lambda | any of the above | [the ECS module](/install/ecs/#lambda-functions) with `watch_lambda`, or any agent with AWS credentials |

## Enrollment codes

An enrollment code (`glsh_enroll_…`) lets an agent register itself: start the agent with the code where a token
would go, and it finds what it can read where it runs, becomes an agent named after it and adds its targets to the
code's environment. Nothing has to be set up in Goliash first.

**Connect a cluster or host** (Settings → Agents and targets) does it in two clicks: pick the platform, then *Get
the command*. The command carries a new code; the page then follows the agents that register with it, the targets
they found and their first report. *Set up by hand* below it still creates a target with your own settings, for an
agent you have or for the server.

From the CLI:

```sh
goliash enroll create -env prod              # a code for one agent, valid 7 days
goliash enroll create -env prod -many        # one code for a fleet, e.g. every Docker host
```

| The agent runs | It adds | Named after | Needs |
| --- | --- | --- | --- |
| In Kubernetes | the cluster | `name` in the Helm values, else `kubernetes` | nothing |
| On ECS, on EC2 with an instance role, or with AWS credentials and `AWS_REGION` | every ECS cluster of its region, each a target of its own | its cluster, else `aws-<region>` | `ecs:ListClusters` (else only its own cluster) |
| In Nomad | the region, through the Nomad agent of its node | `nomad-<region>` | `NOMAD_ADDR` (set by the job file) |
| With a Docker socket | the host, or the swarm on a manager | the host name | `INFO=1` on the socket proxy |
| The same | the region's Lambda functions, as `lambda-<region>` | | `lambda:ListFunctions`, `lambda:GetFunction` (`lambda:GetLayerVersion` for layers' packages) |

On EC2 the agent asks the instance metadata for its region and role, so an agent in Docker on an EC2 instance
needs no AWS settings: give the instance's role the read-only policy (**Add AWS** on the agent's page has the one
command) and the agent adds what the role may read. Its container reaches the role only with an instance metadata
hop limit of 2; the agent says so when it is 1. `GOLIASH_AWS=off` stops it from asking.

The agent looks again every 15 minutes (`GOLIASH_REDISCOVER`, `off` to look only at start) and, when it finds
something new, such as a service the role may now read, enrolls or registers again with it.

In AWS, `GOLIASH_AWS_REGIONS` adds the clusters and functions of more regions (their targets end in the region)
and `GOLIASH_ECS_CLUSTERS` keeps only the named clusters. Each cluster lands in the code's environment; move the
staging one to staging on its target page once, and it stays there.

`GOLIASH_AGENT_NAME` sets the name, and `GOLIASH_TARGETS` declares more targets as a JSON array, e.g. Compose
files: `[{"platform":"compose","name":"shop","compose":{"files":["/srv/shop/compose.yml"]}}]`. Targets the agent
added follow what it reports when it starts again; their environment, and anything else, can still be changed in
Goliash.

The agent keeps no state: it enrolls on every start and gets a new token, which ends the old one.

- **A code for one agent** belongs to the first agent that enrolls with it, which can come back with it at any time.
  It works like a token that the agent exchanges for its own.
- **A code for many agents** (`-many`) tells them apart by what identifies their installation: the cluster's CA, the
  ECS cluster, the Nomad job and region, or the Docker engine or swarm ID. Where none is found, set
  `GOLIASH_AGENT_ID` to a name that stays the same across restarts.
- **Expiry** (7 days by default, `-expires 720h` or `-expires never`) stops new agents only: agents that enrolled
  keep coming back with the code. **Revoking** it (`goliash enroll revoke -id ID`) stops every agent from enrolling
  with it again, and **revoking an agent** keeps it out until it is deleted. `goliash enroll list` shows the codes
  and how many agents used each.

The **Enrollment codes** panel on the agents page lists the codes with their environment, how many agents used
them and until when new agents may join, and revokes them. Targets an agent found show *managed by agent*: their
settings come from the agent on every start, while their environment and interval stay yours to change.

Agents created with `goliash agent create` keep their token and their targets set up in Goliash.

## Managing agents

Create an agent under **Settings → Agents and targets** (or `goliash agent create -name prod-eu`). Its page shows the token once,
with commands to start the agent on a Docker host, Kubernetes, Nomad or as a binary, and then:

- **Status**: online, stale (missed heartbeats), never connected or revoked, its version (with *update available*
  when it is older than the server), host, platforms and the targets it collects.
- **Update**: an agent older than the server shows the command that updates it where it runs, keeping its code or
  token and settings: `helm upgrade --reuse-values`, the Docker one-liner, `docker service update`, the Nomad job
  with `-var version`, or a CloudFormation `update-stack` with the previous values.
- **Rotate token** without a gap: the old token keeps working until the agent first connects with the new one, then
  stops. Rotating an agent that never connected replaces its token at once. CLI: `goliash agent rotate -name N`.
- **Revoke** every token when one leaked or the machine is gone; the agent stops sending data at once.
  CLI: `goliash agent revoke -name N`.
- **Rename** (the token stays) and **delete** an agent once it has no targets.
- **Move a target** to another agent or to the server, or delete it. The agents pick the change up with their next
  configuration poll. Deleting a target removes its running versions from the matrix; the history stays.

`goliash agent list` prints every agent with its status, version and last contact.

## Database

SQLite is the default and suits most installations: one file. For PostgreSQL, set `GOLIASH_DATABASE_URL` to a
`postgres://` URL. Migrations run on start for both.

### Backups

**SQLite** backs up while the server runs: `goliash backup -out /backups` writes a consistent copy
(`goliash-<time>.db`) and, when the secret key is the file next to the database, `goliash-<time>.key` beside it. Or
let the server do it: `GOLIASH_BACKUP_DIR=/data/backups` writes one at start and every day after, keeping
`GOLIASH_BACKUP_KEEP` (default 7). Put the directory on other storage, or copy it off the machine.

To restore, stop the server, put the copy in place as `goliash.db` and its key as `goliash.key` (or keep
`GOLIASH_SECRET_KEY`), and start it.

**PostgreSQL** is backed up with its own tools: `pg_dump -Fc "$GOLIASH_DATABASE_URL" > goliash.dump`, or your
provider's snapshots. Keep `GOLIASH_SECRET_KEY` with them.

### Upgrades

Replace the image or binary and start it: migrations run on start, under a lock when several servers share
PostgreSQL. Take a backup first. Every change is tested by upgrading a database written by the first release and by
the latest release, on SQLite and PostgreSQL, and checking that its services, history and encrypted channel secrets
survive (`scripts/upgrade-test.sh`). Downgrades are not supported: restore the backup instead.

### Logs and health

`GOLIASH_LOG_FORMAT=json` writes one JSON object per line for Loki or Elasticsearch (the agent too). `/healthz`
says whether the server runs and reaches its database (liveness); `/readyz` also answers 503 while it shuts down,
so load balancers stop sending first: set `GOLIASH_DRAIN=5s` to wait that long before closing connections (the Helm
chart does). `/metrics` includes the server's own health; see [metrics](/guide/notifications/#metrics-instead-of-messages).

### Several servers

With PostgreSQL, run as many servers as you like behind one address (the Helm chart's `replicaCount`); the load
balancer needs no sticky sessions. Every server serves the UI, the REST API, MCP and agents. One of them, the
**leader**, also runs the background work: processing snapshots, checking upstreams, sending notifications,
housekeeping and server-side collectors.

- **Failover.** Leadership is a lease in the database that the leader renews every 5 seconds and that lasts 15,
  timed by the database's clock. When the leader crashes, freezes or loses its network, another server takes over
  within about 20 seconds; a leader that comes back after its lease ran out stops its work first. A server that
  shuts down on purpose hands over at once. `goliash_leader` in `/metrics` shows which server leads.
- **Shared state.** Sessions, sign-in tokens, failed sign-in counts, setup links and everything else live in the
  database. New snapshots wake the leader and live updates reach every server's browsers through PostgreSQL
  `LISTEN`/`NOTIFY`. MCP over HTTP keeps no session, so any server answers any request. Migrations run under a lock,
  so servers may start together.
- **What you provide.** Give every server the same `GOLIASH_SECRET_KEY` (a server with another key refuses to
  start rather than misread channel secrets) and the same `GOLIASH_PUBLIC_URL`. The database itself must be highly
  available (a managed PostgreSQL, or Patroni and the like): Goliash keeps no state outside it.

SQLite allows one server.

## Secrets at rest

Notification channel secrets (Slack and webhook URLs, signing secrets) are encrypted with AES-256-GCM. The key comes
from `GOLIASH_SECRET_KEY` or `GOLIASH_SECRET_KEY_FILE`:

```sh
openssl rand -base64 32
```

With SQLite and neither set, the server creates `goliash.key` next to the database. With PostgreSQL, set the key
yourself; without one, secrets are stored unencrypted and the server logs a warning.

:::caution[Back up the key with the database]
Without the key, stored channels cannot be read and must be created again. The server refuses to start when
its key does not open the stored secrets, rather than failing at the first notification.
:::
## Public URL

Set `GOLIASH_PUBLIC_URL` to the address people use, for example `https://goliash.example.com`. Sign-in links and
cookies depend on it; with `https://`, cookies are marked Secure. Put the server behind a reverse proxy or ingress
that terminates TLS.

All settings are listed in [Configuration](/reference/configuration/).

## Clean up

**Agents and targets → Clean up** lists what a workspace keeps for nothing, to delete in bulk: agents not heard from
for a week (with the targets they found themselves), targets nothing has reported for in a week, environments
without targets, services that run nowhere, and mapping rules that match nothing (their service is gone, or they
name a Lambda runtime or a database engine, which maps by name). Only what the page still offers when you press
delete goes: an agent that came back meanwhile stays.

---

# Docker and Compose
Source: /install/docker/

> Run the Goliash server with Docker Compose and watch Docker hosts and Compose stacks through a read-only socket proxy.

## Server

The repository's `docker-compose.yml` runs the published image with SQLite on a volume:

```sh
curl -fsSLO https://raw.githubusercontent.com/pipozzz/goliash/main/docker-compose.yml
docker compose up -d
docker compose logs goliash | grep link=        # open it to create your account
```

Open the server and create the first account: the sign-in page offers it for 15 minutes after the start, later
use the setup link from the log. On a laptop the server is `http://localhost:8080`; **Connect → Docker host** then gives a
command that reaches it from the agent's container through `host.docker.internal`, so a trial needs no other
address. Agents on other machines need `GOLIASH_PUBLIC_URL` set to an address they can reach. `GOLIASH_VERSION=1.0.0` pins the image (the tag `1`
follows 1.x). Set `GOLIASH_PUBLIC_URL` to the address people use, and put a reverse proxy with TLS in front for
anything beyond a trial.

Without Compose:

```sh
docker run -d --name goliash -p 8080:8080 -v goliash-data:/data \
  -e GOLIASH_PUBLIC_URL=https://goliash.example.com \
  ghcr.io/pipozzz/goliash:latest
```

The image's health check command is `goliash healthcheck`.

## Watch the host the server runs on

No agent is needed. The `watch-host` profile adds a
[docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy) that allows only `GET` on containers and
images:

```sh
docker compose --profile watch-host up -d
docker compose exec goliash goliash env create -name prod -position 30
docker compose exec goliash goliash target create -env prod -platform docker -name this-host \
  -settings '{"docker":{"docker_host":"tcp://socket-proxy:2375"}}'
```

## Agent on another host

`deploy/docker/goliash-agent.yml` runs an agent with its own socket proxy on any Docker host. **Connect** →
**Docker** gives these commands with the server and a code filled in; the files stay in `~/goliash-agent`:

```sh
mkdir -p ~/goliash-agent && cd ~/goliash-agent
curl -fsSLo compose.yml https://raw.githubusercontent.com/pipozzz/goliash/main/deploy/docker/goliash-agent.yml
(umask 077; printf '%s\n' GOLIASH_SERVER_URL=https://goliash.example.com GOLIASH_AGENT_TOKEN=glsh_enroll_… > .env)
docker compose up -d
```

`GOLIASH_AGENT_TOKEN` takes the code from **Connect** (or `goliash enroll create -env prod`). Change settings in
`~/goliash-agent/compose.yml` and run `docker compose up -d` there again. Its log: `docker compose logs agent`; to
remove it: `docker compose down`.

The agent finds by itself the containers and Compose projects, the host's OS and packages (the host's root is
mounted read-only at `/host`; remove that line to leave them out), and on EC2 the AWS services the instance's role
may read. It looks again every 15 minutes.

With an [enrollment code](/install/#enrollment-codes) the agent adds the host as a `docker` target itself. With an
agent token instead, create the target with `{"docker":{"docker_host":"tcp://socket-proxy:2375"}}`.

To check private registries with the host's `docker login`, mount its Docker config into the agent (read-only) and
point `DOCKER_CONFIG` at it:

```yaml
services:
  agent:
    environment:
      DOCKER_CONFIG: /etc/goliash-agent/docker
    volumes:
      - /root/.docker/config.json:/etc/goliash-agent/docker/config.json:ro
```

Docker Desktop keeps logins in a credential helper instead; there, give the agent a credential per registry host
(see [Private registries](/reference/collectors/#private-registries)).

## Track only some Compose projects

To watch one stack instead of the whole host, list its Compose projects:

```json
{"docker": {"docker_host": "tcp://socket-proxy:2375", "projects": ["shop"]}}
```

## Track a Compose file without access to Docker

A `compose` target reads Compose files as declared, without a Docker engine, socket proxy or agent. It suits stacks
kept in Git, or deployed through a platform such as Dokploy, Nomploy or Coolify where Goliash cannot reach the host:

```sh
goliash target create -env prod -platform compose -name shop \
  -settings '{"compose":{"files":["https://raw.githubusercontent.com/acme/infra/main/shop/compose.yaml"]}}'
```

- **Several files** override each other in order, like `docker compose -f compose.yaml -f compose.prod.yaml`.
- **Variables:** `${TAG:-1.2.3}` uses its default; set values with `"variables": {"TAG": "1.3.0"}`.
- **Private repositories:** give the target a credentials reference; the URL is fetched with
  `Authorization: Bearer <credential>` (a GitHub or GitLab token).
- **What it reports:** every service with an `image`, with its declared replicas. Services built from source without
  an `image` are skipped. The project is `"project"` from the settings, else the file's `name`.
- **Files on disk:** an agent can read local files from directories its operator lists in `GOLIASH_COMPOSE_DIRS`, for
  example `/srv/stacks`. The server itself reads URLs only, and never link-local addresses such as cloud metadata.

Declared versions are what the file says, not what runs. Add a `docker` target for the same environment as well,
and Goliash compares the two: `declared` drift shows where the host runs something other than what Git declares.
See [Versions and drift](/guide/versions/#declared-versus-running).

## What the Docker collector reports

- **Compose services**: containers with Compose labels are grouped by project and service. The project is the
  namespace and the replica count is the number of running containers.
- **Other containers** are reported one by one, by container name.
- **Digests**: the registry digest of each running image, when the proxy allows image reads (`IMAGES=1`).
- **Skipped**: containers that belong to Swarm services (use a [Swarm target](/install/swarm/) for those) and one-off
  `docker compose run` containers.

A `goliash.service` label on a Compose service names its Goliash service; otherwise it waits in the inbox.
Compose service names like `db` or `cache` are deliberately not mapped automatically, because they would merge
unrelated projects.

---

# Amazon ECS and Lambda
Source: /install/ecs/

> Watch Amazon ECS clusters, Lambda functions and private ECR repositories with the Goliash agent and a read-only IAM task role.

## Try it in one line

**Connect → Amazon ECS → Get the command** in Goliash gives this with your server and a fresh enrollment code. It
deploys the agent with CloudFormation into the default VPC, as a Fargate service in a small cluster of its own:

```sh
curl -fsSLo goliash-agent.cfn.yaml https://raw.githubusercontent.com/pipozzz/goliash/main/deploy/ecs/goliash-agent.cfn.yaml && \
aws cloudformation deploy --stack-name goliash-agent --template-file goliash-agent.cfn.yaml --capabilities CAPABILITY_IAM \
  --parameter-overrides ServerURL=https://goliash.example.com EnrollCode=glsh_enroll_… \
  Subnets=$(aws ec2 describe-subnets --filters Name=default-for-az,Values=true --query 'Subnets[].SubnetId' --output text | tr '\t' ,)
```

Within a minute the agent registers and adds every ECS cluster of the region, each as a target, and the region's
Lambda functions (`WatchLambda=true`, the default). Its log: `aws logs tail /ecs/goliash-agent --follow`. Remove it
all with `aws cloudformation delete-stack --stack-name goliash-agent`.

## Add AWS to an agent you already run

An agent on a Docker host or in a Kubernetes cluster can watch AWS too: **Agents → the agent → Add AWS to this
agent**.

**On EC2 with an instance role** (agent 1.153.0 or newer) there is one step. The agent reads its region and role from
the instance metadata and reports them; the page then shows one `aws iam put-role-policy` command with that role
and the read-only policy in it (run it anywhere with IAM access, e.g. AWS CloudShell). Within 15 minutes the agent
looks again and adds what the role may now read; nothing to set on the agent, no restart. An instance without a
role gets the commands that create one and attach it. If the container gets no metadata token, the page says so
and gives the command that raises the hop limit to 2.

**Otherwise** pick the region, the services and how the agent gets credentials (a read-only IAM user's keys, a
pod's role, or a profile). The page writes:

1. the read-only IAM policy for exactly those services, and the `aws` commands that give it to a new IAM user or a
   role;
2. the lines for `~/goliash-agent/compose.yml` (or the Helm values);
3. the command that restarts it.

Then the agent finds what the credentials may read:

- **With an enrollment code**, it adds those targets by itself.
- **With a token**, it shows them on its page under **Found here, not watched**: tick them, pick an environment,
  and **Add**. The Agents list says when an agent found something (*3 found*).
- **Not collected** says what it could not read and why. A missing permission is named with the IAM actions to
  allow.

The agent keeps its identity (the Docker host or cluster) when it also gets AWS credentials.

## CloudFormation

`deploy/ecs/goliash-agent.cfn.yaml` creates the task and execution roles, a Secrets Manager secret holding the
code, a log group and the service. Parameters:

| Parameter | Default | Meaning |
| --- | --- | --- |
| `ServerURL`, `EnrollCode` | | Where to report, and the code from Connect (or an agent token) |
| `Subnets` | | Where the task runs; private subnets need a NAT gateway |
| `AssignPublicIp` | `ENABLED` | `DISABLED` in private subnets |
| `Cluster` | new cluster | Run in a cluster you have instead |
| `SecurityGroups` | the VPC's default | The agent only connects out |
| `Version` | `latest` | Agent image tag; the server's version fits best |
| `WatchLambda` | `true` | Also read Lambda functions and aliases |
| `Regions`, `Clusters` | | More regions; only these clusters |

## Terraform

The Terraform module in `deploy/ecs/goliash-agent` runs the agent as a Fargate service. Its task role may only call
`ecs:List*`, `ecs:Describe*`, `ecr:ListImages`, `elasticloadbalancing:DescribeTargetGroups` and `DescribeLoadBalancers` (which load balancers face the internet), and for signatures and SBOMs of ECR images
`ecr:GetAuthorizationToken`, `ecr:BatchGetImage` and `ecr:GetDownloadUrlForLayer`.

```hcl
module "goliash_agent" {
  source             = "github.com/pipozzz/goliash//deploy/ecs/goliash-agent"
  cluster_arn        = aws_ecs_cluster.tools.arn
  server_url         = "https://goliash.example.com"
  token_secret_arn   = aws_secretsmanager_secret.goliash_agent_token.arn
  subnet_ids         = module.vpc.private_subnets
  security_group_ids = [aws_security_group.egress_only.id]
}
```

The agent needs outbound HTTPS only. With an [enrollment code](/install/#enrollment-codes) in the secret it adds every
ECS cluster of its region by itself, each as a target of its own, so the staging cluster can go to staging and the
production one to production. `regions = ["us-east-1"]` adds the clusters of more regions and
`clusters = ["prod", "staging"]` keeps only some. With an agent token, create an `ecs` target for it with
`{"ecs":{"region":"eu-west-1"}}`, and optionally `"clusters": [...]` to limit it to some clusters.

## Lambda functions

Set `watch_lambda = true` and the task role may also call `lambda:ListFunctions`, `lambda:GetFunction` and
`lambda:ListAliases`. An agent
that enrolls with a code then adds the region's functions as a `lambda` target, named `lambda-<region>`; any agent
with AWS credentials that may list functions does the same (`GOLIASH_LAMBDA=off` stops it). With a token, create the
target with `{"lambda":{"region":"eu-west-1"}}`, optionally with `"name_prefixes": ["shop-"]`.

Aliases named like environments (`dev`, `staging`, `prod`) put each function's versions in those environments.
Map other names with `GOLIASH_LAMBDA_ALIASES=live=prod,canary=-` on an enrolling agent, or `alias_environments` in
the target's settings.

Container functions show their image like any service. A .zip function shows its runtime as a version
(`python 3.12`), with *new release* when AWS offers a newer runtime and *end of life* before AWS deprecates it. See
[Collectors](/reference/collectors/#target-settings).

## RDS and Aurora databases

With `WatchRDS=true` (CloudFormation, the default) or `watch_rds = true` (Terraform) the task role may also call
`rds:DescribeDBInstances`, `rds:DescribeDBClusters`, `rds:DescribeDBEngineVersions` and
`rds:DescribePendingMaintenanceActions`, and the agent adds the
region's databases as an `rds` target named `rds-<region>` (`GOLIASH_RDS=off` stops it). With a token, create the
target with `{"rds":{"region":"eu-west-1"}}`.

Each database is a service named like the instance, or the Aurora cluster with its instances, so it gets what any
service gets in the matrix, Updates, notifications and reports:

- **Its engine and version**, shown as `rds.amazonaws.com/postgres:16.4`. PostgreSQL, MySQL, MariaDB, Aurora
  PostgreSQL and Aurora MySQL are read. Aurora MySQL shows its own version (3.05.2 rather than
  8.0.mysql_aurora.3.05.2), which its support follows.
- **The versions RDS offers** in the region, as its upstream releases. A newer minor or major version shows as
  *upstream* drift and on Updates.
- **End of standard support** (from endoflife.date), as *end of life*. After that date AWS moves the database to
  RDS Extended Support and charges for it per vCPU and hour, unless the upgrade happens first.
- **Public accessibility.** A database with `PubliclyAccessible` is *exposed*, with its endpoint, and its upgrades
  come first.

Tag a database (or cluster) `goliash.ignore=true` to leave it out, and `goliash.env=prod` to place it in an
environment other than the target's.

**Patches waiting.** The maintenance RDS has pending for a database (OS patches, minor upgrades, certificate
rotation), ElastiCache's service updates (security updates first) and OpenSearch's service software updates show
as **patch pending** on the service page and on Updates, with what waits and when AWS applies it. The CSV and
`GET /api/v1/updates` have them as `maintenance`.

## ElastiCache caches

With `WatchElastiCache=true` (CloudFormation, the default) or `watch_elasticache = true` (Terraform) the task role
may also call `elasticache:Describe*` and `elasticache:ListTagsForResource`, and the agent adds the region's caches
as an `elasticache` target named `elasticache-<region>` (`GOLIASH_ELASTICACHE=off` stops it). With a token:
`{"elasticache":{"region":"eu-west-1"}}`.

Each replication group, standalone cluster (Memcached) and serverless cache is a service, its engine shown as
`elasticache.amazonaws.com/redis:7.1.0` (or `valkey`, `memcached`):

- **The versions ElastiCache offers** are its upstream releases, so newer engine versions show on Updates.
- **End of life** comes from endoflife.date: ElastiCache's own dates for Redis OSS (after which AWS charges for
  Extended Support), upstream Valkey and Memcached for those.
- **During an engine upgrade** the group shows both versions, like a rollout.
- **Tags:** `goliash.ignore=true` and `goliash.env` on the cache work as on databases. Without
  `elasticache:ListTagsForResource` caches still count, without their tags.

## EKS clusters

With `WatchEKS=true` (CloudFormation, the default) or `watch_eks = true` (Terraform) the task role may also call
`eks:ListClusters`, `eks:DescribeCluster`, `eks:ListNodegroups`, `eks:DescribeNodegroup` and
`eks:DescribeClusterVersions`, and the agent adds the region's clusters as an `eks` target named `eks-<region>`
(`GOLIASH_EKS=off` stops it). With a token: `{"eks":{"region":"eu-west-1"}}`.

This watches the clusters themselves; the workloads inside a cluster come from an agent in it (see
[Kubernetes](/install/kubernetes/)). Each cluster is a service named like it:

- **Its control plane and each managed node group** report their Kubernetes version as
  `eks.amazonaws.com/kubernetes:1.33`. A node group left a version behind shows as a second version of the
  cluster, with its node AMI release in the workload's labels.
- **The versions EKS offers** are its upstream releases, so the next Kubernetes version shows on Updates.
- **End of standard support** comes from endoflife.date (`amazon-eks`), as *end of life*. After it, AWS keeps the
  cluster in extended support and charges for it per cluster and hour, unless the cluster's upgrade policy says
  otherwise.
- **A public API endpoint** open to `0.0.0.0/0` makes the cluster *exposed*.
- **Tags:** `goliash.ignore=true` and `goliash.env` on the cluster work as on databases; node groups take the
  cluster's tags, then their own.

Self-managed nodes and Fargate profiles are not read.

## OpenSearch Service domains

With `WatchOpenSearch=true` (CloudFormation, the default) or `watch_opensearch = true` (Terraform) the task role may
also call `es:ListDomainNames`, `es:DescribeDomains`, `es:ListTags` and `es:ListVersions`, and the agent adds the
region's domains as an `opensearch` target named `opensearch-<region>` (`GOLIASH_OPENSEARCH=off` stops it).

Each domain is a service, its engine shown as `opensearch.amazonaws.com/opensearch:2.13`, or
`opensearch.amazonaws.com/elasticsearch:7.10` for legacy Elasticsearch:

- **The versions the service offers** are its upstream releases.
- **End of standard support** comes from endoflife.date (`amazon-opensearch`, `amazon-elasticsearch`), after which
  AWS charges for Extended Support.
- **A domain outside a VPC** (public endpoint) is *exposed*.
- **A service software update AWS has waiting** (patches, often security fixes) shows as *patch pending*.

## MSK clusters

With `WatchMSK=true` (CloudFormation, the default) or `watch_msk = true` (Terraform) the task role may also call
`kafka:ListClustersV2` and `kafka:ListKafkaVersions`, and the agent adds the region's provisioned clusters as an
`msk` target named `msk-<region>` (`GOLIASH_MSK=off` stops it). Serverless clusters have no Kafka version to keep up
and are left out.

- **Each cluster** shows its Kafka version as `kafka.amazonaws.com/kafka:3.6.0`. MSK's `3.7.x` (MSK applies the
  patches itself) reads as 3.7, and its `.kraft` and `.tiered` variants as the Kafka version they run.
- **The versions MSK offers**, and end of support from endoflife.date (`amazon-msk`), work as for the others.
- **Brokers with public access** make the cluster *exposed*.

## Leaving things out

- `goliash.ignore=true` as a tag on an ECS service, Lambda function, RDS database, ElastiCache cache, EKS cluster,
  OpenSearch domain or MSK cluster keeps the agent from reporting it.
- `Clusters` (`GOLIASH_ECS_CLUSTERS`) and `GOLIASH_ECS_EXCLUDE` choose the clusters.
- `WatchLambda`, `WatchRDS`, `WatchElastiCache`, `WatchEKS`, `WatchOpenSearch` and `WatchMSK` set to `false`
  (`GOLIASH_LAMBDA`, `GOLIASH_RDS`, `GOLIASH_ELASTICACHE`, `GOLIASH_EKS`, `GOLIASH_OPENSEARCH`, `GOLIASH_MSK` set to
  `off`) leave that service out.
- In Goliash, an ignore rule (Inbox → **Ignore this image**) drops what the agent still reports.

## Upgrading the stack

**Agents → Update** gives the command. It fetches the template of the new version, so the task role gains what the
new version reads (RDS, ElastiCache, EKS, OpenSearch, MSK, Lambda function URLs), and keeps your parameters.

## What the ECS collector reports

For every cluster in the region: services, their running tasks and the containers of those tasks' task
definitions, with digests. A deployment in progress shows both task definitions' versions.

## Private ECR repositories

Images in private ECR repositories (`<account>.dkr.ecr.<region>.amazonaws.com/…`) are checked by the agent through
the ECR API with the task role. Repositories in other accounts need a repository policy that allows
`ecr:ListImages` for the agent's role.

## Server

The server can run anywhere the team reaches it. On ECS, use PostgreSQL (`GOLIASH_DATABASE_URL`) rather than
SQLite, and set `GOLIASH_SECRET_KEY` from Secrets Manager.

---

# Kubernetes
Source: /install/kubernetes/

> Install Goliash and its agent on Kubernetes with signed Helm charts and a read-only ClusterRole.

The charts are published with every release as signed OCI artifacts in GHCR:
`oci://ghcr.io/pipozzz/charts/goliash` and `oci://ghcr.io/pipozzz/charts/goliash-agent`. The chart version equals the
Goliash version. The sources are in the repository under `deploy/helm`.

## Server

```sh
helm install goliash oci://ghcr.io/pipozzz/charts/goliash -n goliash --create-namespace \
  --set publicURL=https://goliash.example.com \
  --set ingress.enabled=true --set ingress.hosts[0].host=goliash.example.com
kubectl -n goliash logs deploy/goliash | grep link=      # the setup link for the first account
```

The chart runs one replica with SQLite on a persistent volume. For PostgreSQL, set `database.url` or
`database.existingSecret`.

SMTP, OIDC and the secret key come from a secret named in `envFromSecret`:

```sh
kubectl -n goliash create secret generic goliash-env \
  --from-literal=GOLIASH_SECRET_KEY="$(openssl rand -base64 32)" \
  --from-literal=GOLIASH_SMTP_ADDR=smtp.example.com:587 \
  --from-literal=GOLIASH_SMTP_FROM=goliash@example.com
helm upgrade goliash oci://ghcr.io/pipozzz/charts/goliash -n goliash --reuse-values --set envFromSecret=goliash-env
```

With SQLite the secret key is otherwise created next to the database on the volume. With PostgreSQL and no key in
`envFromSecret`, the chart creates one in a Secret named `<release>-goliash-secret-key` (`secretKey.existingSecret`
names your own instead). Upgrades keep it, and so does `helm uninstall`: the encrypted channel secrets and private
registry credentials in the database need it, so back it up with the database.

With PostgreSQL the chart may run several servers: `--set replicaCount=2` (one leads the background work; see
[Several servers](/install/#several-servers)). Updates then roll one pod at a time and a PodDisruptionBudget keeps
one running. With SQLite the chart refuses more than one.

### Watch the cluster the server runs in

To watch only the cluster Goliash runs in, skip the agent. Install the server with `--set collectInCluster=true`,
which gives the server a read-only ClusterRole, and create a `kubernetes` target without an agent.

### Prometheus, alerts and Grafana

`metrics.serviceMonitor.enabled`, `metrics.prometheusRule.enabled` and `metrics.grafanaDashboard.enabled` create a
ServiceMonitor, the alert rules and the Grafana dashboard for the Prometheus Operator; see
[Grafana dashboard](../../guide/grafana/#kubernetes).

## Agent

**Connect → Kubernetes → Get the command** gives the one command, with an enrollment code
(or `goliash enroll create -env prod`):

```sh
helm upgrade --install goliash-agent oci://ghcr.io/pipozzz/charts/goliash-agent -n goliash --create-namespace \
  --set serverURL=https://goliash.example.com --set token.value=glsh_enroll_… --set name=prod-eu
```

The agent registers itself and adds the cluster as a target. To keep the code out of Helm's values, put it in a
secret and pass `--set token.existingSecret=<secret>` instead. With an agent token (`goliash agent create`), add a
`kubernetes` target for it, for example with `{"kubernetes":{"exclude_namespaces":["kube-system"]}}`. An agent
that registered itself reads the same from its environment:
`--set env.GOLIASH_EXCLUDE_NAMESPACES=kube-system\,sandbox` (or `GOLIASH_INCLUDE_NAMESPACES` for only some). A
workload labelled `goliash.ignore=true` is never reported.

The agent's ClusterRole allows only `get`, `list` and `watch` on Deployments, ReplicaSets, StatefulSets,
DaemonSets, CronJobs, Jobs and Pods, and `list` on Services, Ingresses and HTTPRoutes (which workloads can be reached
from outside). It watches for changes and sends a snapshot shortly after a rollout, and a full
snapshot on the poll interval.

### Private registries

The agent checks the tags of private registries with credentials from its own environment, never from the server.
The simplest is to reuse what the cluster already pulls with:

- **An existing image pull secret** (`kubernetes.io/dockerconfigjson`) in the agent's namespace becomes the agent's
  Docker config. No extra permissions:

  ```sh
  helm upgrade goliash-agent oci://ghcr.io/pipozzz/charts/goliash-agent -n goliash --reuse-values \
    --set registry.dockerConfigSecret=regcred
  ```

- **The pull secrets your workloads use**, read where they are: with `rbac.readPullSecrets=true` the agent reads the
  `imagePullSecrets` that running pods reference, and nothing else. Kubernetes RBAC cannot narrow `get` to those
  secrets, though: the role allows reading any secret by name, so choose this only where that is acceptable.

Or give credentials one by one: put them in a secret and name it in `credentialsSecret`; every key becomes a file
in `/etc/goliash-agent/credentials`. The key is the registry host, for example `registry.example.com` with
`user:password` or a token. These come first.

On GKE and AKS, the agent can use its pod's cloud identity for Google Artifact Registry and Azure Container
Registry, with no secret: bind its service account to a Google service account with
`roles/artifactregistry.reader` (Workload Identity), or to an Azure managed identity with `AcrPull` (label the pod
`azure.workload.identity/use: "true"` through `podLabels`, and annotate the service account with
`azure.workload.identity/client-id`).

For Amazon ECR on EKS, give the agent an IAM role with `ecr:ListImages` (and `ecr:GetAuthorizationToken`,
`ecr:BatchGetImage`, `ecr:GetDownloadUrlForLayer` for signatures and SBOMs) through IRSA:

```sh
helm upgrade goliash-agent oci://ghcr.io/pipozzz/charts/goliash-agent -n goliash --reuse-values \
  --set-string 'serviceAccount.annotations.eks\.amazonaws\.com/role-arn=arn:aws:iam::123456789012:role/goliash-agent'
```

## Verify the charts

```sh
cosign verify ghcr.io/pipozzz/charts/goliash:0.2.0 \
  --certificate-identity-regexp '^https://github.com/pipozzz/goliash/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
```

---

# Nomad
Source: /install/nomad/

> Run Goliash on Nomad with a job or the Nomad pack, and watch jobs with a list-jobs and read-job ACL token.

## One command with nomad-pack (Nomploy)

The `goliash` pack runs the server with SQLite on a volume and watches the Nomad cluster it runs on, read-only,
without an agent. The matrix fills on its own:

```sh
nomad-pack registry add nomploy https://github.com/Nomploy/nomad-packs
nomad-pack run goliash --registry=nomploy \
  --var owner_email=you@example.com --var public_url=https://goliash.example.com
```

The first start logs a one-time sign-in link for `owner_email` in the task logs. With Nomad ACLs on, pass a token
with `list-jobs` and `read-job` as `nomad_token`. All variables are in the
[pack README](https://github.com/pipozzz/goliash/tree/main/deploy/nomad/pack/goliash).

## Server

`deploy/nomad/goliash-server.nomad.hcl` runs the server. SQLite lives in the host volume `goliash-data`, which
you declare in the client configuration. For PostgreSQL, put a URL in the job's variable:

```sh
nomad var put nomad/jobs/goliash database_url=postgres://…   # optional
nomad job run deploy/nomad/goliash-server.nomad.hcl
```

## Agent

The agent reads the cluster through the Nomad API with a token that has the `list-jobs` and `read-job` capabilities:

```sh
nomad acl policy apply goliash-read - <<<'namespace "*" { capabilities = ["list-jobs", "read-job"] }'
nomad acl token create -name goliash-agent -policy goliash-read      # copy the secret ID
nomad var put nomad/jobs/goliash-agent token=glsh_enroll_… nomad_token=<secret ID>
nomad job run -var server_url=https://goliash.example.com -var version=1.14.0 deploy/nomad/goliash-agent.nomad.hcl
```

`token` is the code from **Connect** (or `goliash enroll create -env prod`): the agent registers itself and adds the
region as a target, read through the Nomad agent on its node. The job passes the Nomad token to the agent as
`GOLIASH_CREDENTIAL_NOMAD`. With an agent token (`glsh_agent_…`) instead, create a `nomad` target with credentials
reference `nomad` and settings like `{"nomad":{"address":"http://nomad.service.consul:4646"}}`.

Run one agent per token: the job is a `service` job with one instance, not a `system` job.

## What the Nomad collector reports

One workload per job, with a container per task (`group/task`) and its image, counted by running allocations.
During a deployment the old and new job versions show side by side. Optional settings: `region` and `namespaces`
(empty means every namespace the token can read).

---

# Docker Swarm
Source: /install/swarm/

> Deploy Goliash on Docker Swarm with a stack file and watch services through a read-only socket proxy.

## Server

`deploy/swarm/goliash-server.yml` runs the server with SQLite on a local volume, so it is pinned to one node:

```sh
docker stack deploy -c deploy/swarm/goliash-server.yml goliash
docker service logs goliash_server 2>&1 | grep link=      # the setup link for the first account
```

Edit `GOLIASH_PUBLIC_URL` in the file first. For PostgreSQL, set `GOLIASH_DATABASE_URL` and drop the placement
constraint.

## Agent

`deploy/swarm/goliash-agent.yml` runs the agent with a docker-socket-proxy that allows only `GET` on services,
tasks and nodes. The proxy runs on a manager node, because the services and tasks APIs answer only there.

```sh
printf '%s' 'glsh_agent_…' | docker secret create goliash_agent_token -
GOLIASH_SERVER_URL=https://goliash.example.com docker stack deploy -c deploy/swarm/goliash-agent.yml goliash-agent
```

Create a `swarm` target for the agent with `{"swarm":{"docker_host":"tcp://socket-proxy:2375"}}`.

When the server runs in the same swarm, uncomment the `goliash` network in the agent file and use
`GOLIASH_SERVER_URL=http://goliash_server:8080`.

## What the Swarm collector reports

Services with their desired and running replicas, and the image of every running task, digest included. During a
rolling update both versions show. The stack name (`com.docker.stack.namespace`) is the namespace.

---

# Agent protocol
Source: /reference/agent-protocol/

> The HTTP protocol between goliash-agent and the server: registration, configuration, heartbeats and snapshots.

The protocol between `goliash-agent` and the server is described in
[`api/agent-v1.yaml`](https://github.com/pipozzz/goliash/blob/main/api/agent-v1.yaml) and licensed Apache-2.0, so
you can write your own agent or collector. The server validates every request against it.

- The agent only makes outbound HTTPS requests; the server never connects to the agent.
- Every request carries `Authorization: Bearer glsh_agent_…`. One token identifies one agent in one workspace and
  may only send data and read that agent's configuration.
- The agent sends **full state** (snapshots), never events. The server compares consecutive snapshots of a target.
- Credentials stay with the agent. The server only sends a `credentials_ref` name, which the agent resolves
  locally.

Call order: `enroll` on start when the agent has an enrollment code instead of a token, then `register`, then `config` every minute (with an ETag), a `snapshot` per target on its interval
(Kubernetes also shortly after changes), a `heartbeat` every minute, and `registry-results` on the registry check
interval. While the server is unreachable, the agent buffers up to 200 snapshots on disk.

A registry check may carry `resolve`: running digests of images on moving tags, each with the tags it may be. The
agent compares them (`HEAD` first, platform manifests of the two newest candidates only when needed) and answers in
`resolved`. Choosing candidates stays on the server; the agent only compares digests. Older agents ignore the field.

---

# REST API
Source: /reference/api/

> The Goliash REST API: matrix, services, drift, history, promotions, delivery, inventory and hygiene, with OpenAPI.

The API under `/api/v1` reads the matrix, services, environments, targets, history and drift, and creates
acknowledgements.

- **Authentication:** `Authorization: Bearer glsh_api_…` with an API token (Users → API tokens, or
  `goliash token create`), or a browser session. A token belongs to one workspace and is a viewer (reads) or a
  member (also acknowledges).
- **Errors:** `application/problem+json` (RFC 9457).
- **Description:** [`api/public-v1.yaml`](https://github.com/pipozzz/goliash/blob/main/api/public-v1.yaml), also
  served by every server at `/api/v1/openapi.yaml`. A test checks every response against it.

```sh
curl -s -H "Authorization: Bearer $GOLIASH_TOKEN" https://goliash.example.com/api/v1/drifts | jq
curl -s -X POST -H "Authorization: Bearer $GOLIASH_TOKEN" -H 'Content-Type: application/json' \
  -d '{"service":"postgres","kind":"release","until_version":"17.0"}' \
  https://goliash.example.com/api/v1/acks
```

Prometheus metrics are at `GET /metrics`, with the same authentication; see
[Notifications](/guide/notifications/#metrics-instead-of-messages).

---

# CLI
Source: /reference/cli/

> Every goliash command: server, agents, targets, users, tokens, matrix, history, inventory and more.

The server binary is also its command-line tool. It works directly on the database, so run it where the server
runs, for example with `docker compose exec goliash goliash …` or `kubectl exec deploy/goliash -- goliash …`.

```text
Usage:
  goliash [serve] [flags]                 run the server
  goliash env create -name NAME [-position N]
  goliash agent create -name NAME         prints the agent token once
  goliash agent list                      agents with status, version and last contact
  goliash agent rotate -name NAME         new token; the old one works until the agent uses the new one
  goliash agent revoke -name NAME         every token of the agent stops working
  goliash enroll create -env NAME [-many] [-expires 168h|never]
                                          prints an enrollment code once: agents started with it register themselves
  goliash enroll list                     enrollment codes with their environment and agents
  goliash enroll revoke -id ID            no agent can enroll with the code any more
  goliash target create -agent NAME -env NAME -platform kubernetes|ecs|nomad|swarm|docker|compose|lambda -name NAME [-settings JSON] [-poll SECONDS]
  goliash matrix                          service × environment versions
  goliash events [-service NAME] [-limit N]
  goliash drift                           open drifts
  goliash inventory [-at T] [-environment E] [-format table|csv|cyclonedx]
                                          every running container with image and digest (audits, SBOM)
  goliash images SERVICE                  its running images: size against budget, base, signed, SBOM, provenance, last layer change
  goliash licenses [-class attention|all|CLASS] [-environment E] [-os] [-fail-on CLASSES]
                                          licenses of packages in running images, copyleft and unknown first;
                                          -fail-on network-copyleft,source-available exits 1 when one runs
  goliash check [-service NAME]           check upstream registries now
  goliash service set -name NAME [-upstream REPO] [-owner O] [-app A] [-kind own|third_party|auto]
                      [-track patch|minor|major] [-pin-major N] [-tag-filter REGEXP] [-prerelease]
  goliash channel create -type slack|webhook|email|push -name NAME [-url URL] [-secret S] [-to a@b,c@d]
                         [-smtp-addr HOST:PORT -smtp-from ADDR [-smtp-username U] [-smtp-tls starttls|tls|none]]
  goliash channel test -name NAME
  goliash notify create -channel NAME [-events new_release,drift_detected] [-mode instant|daily|weekly]
                        [-services a,b] [-owners x] [-apps webshop] [-envs prod] [-min-jump minor] [-digest-hour 8]
                        [-timezone Europe/Bratislava] [-quiet 22-7]
  goliash badges reset                    invalidate every badge address handed out
  goliash service rename -name NAME -to NAME [-merge]   rename a service, or join it with the one named
  goliash app rename -from NAME [-to NAME] show an application under another name (an existing one merges)
  goliash team rename -from NAME -to NAME   rename a team on all its services
  goliash ack -service NAME -kind release|drift [-until-version 2.1.0] [-for 336h] [-env prod]
  goliash workspace create -name N -slug S [-envs]   a workspace per client or team
  goliash workspace list
  goliash user grant -email E -role viewer|member|admin|none   access to the -workspace
  goliash user create -email E [-role owner|admin|member|viewer] [-name N]
  goliash user password -email E [-remove]   set a password (asked for, or one line on stdin)
  goliash user 2fa -email E -reset        turn two-factor sign-in off (lost phone, no recovery codes)
  goliash login-link -email E             one-time sign-in link (creates the first user as owner)
  goliash token create -name N [-role viewer|member] [-expires 90d]   API token for /api/v1, /metrics and /mcp (shown once)
  goliash token list
  goliash token revoke -name N
  goliash rule create -match image_repo|workload_name|label|ignore -pattern REGEXP [-service NAME] [-priority N]
  goliash backup -out DIR                 SQLite: a consistent copy of the database (and goliash.key) while the server runs
  goliash healthcheck                     exit 0 when the local server answers /healthz (container health checks)
  goliash try [-listen :8080] [-public] [-reset 1h]
                                          a throwaway server with example data and a link that signs you in;
                                          -public makes it a read-only demo for everyone, -reset restarts it fresh
  goliash demo                            fill the workspace with three weeks of example data
  goliash version

Every command takes -database (env GOLIASH_DATABASE_URL, default goliash.db) and
-workspace SLUG (env GOLIASH_WORKSPACE, default: the first workspace).
```

Changes made with the CLI are recorded in the audit log like changes in the UI.

## Ask Goli from a terminal

`goliash ask` works from anywhere, not on the database: it asks a server's [Goli](/guide/goli/) and prints the
answer as it is written. What Goli looks at goes to stderr, so the answer alone can be piped into a ticket, a commit
message or a chat.

Sign in once through the browser; no token to create or copy:

```sh
goliash login https://goliash.example.com   # opens the browser: sign in, Allow
goliash ask "what changed in prod in the last two hours?"
goliash ask "brief me on upgrading postgres in prod to 17" > postgres-17.md
goliash ask -i                  # a conversation: ask, then ask on; an empty line or Ctrl-D ends it
goliash logout                  # ends the sign-in, on the server too
```

`goliash login` signs the CLI in with the server's OAuth, as you and read only: it shows under **Your account →
Apps with access** like any app. The tokens are kept in your config directory (`~/.config/goliash/credentials.json`
on Linux, `~/Library/Application Support/goliash/` on macOS; `GOLIASH_CREDENTIALS` moves it), readable by you
only, and renewed as they run out; the last server signed in to is the default. On a machine without a browser,
`-no-browser` prints the address to open elsewhere (the answer still comes back to this machine's loopback, so
open it on the same machine, e.g. through SSH port forwarding).

Scripts and CI use an API token instead, which wins over a sign-in:

```sh
export GOLIASH_URL=https://goliash.example.com GOLIASH_TOKEN=glsh_api_…   # goliash token create -name cli
```

Goli sees what you, or the token, may see. Scripts and bots can do the same with `POST /goli/ask` and
`Authorization: Bearer glsh_api_…`: the body is `{"history": [{"role": "user", "text": "…"}]}`, and the answer
comes back as server-sent events (`text`, `tool`, `done`, `error`). The hourly and daily limits count per person or
token.

## Examples

```sh
goliash env create -name staging -position 20
goliash enroll create -env prod                         # prints a code for one agent once
goliash agent create -name prod-eu                      # prints the token once
goliash target create -agent prod-eu -env prod -platform kubernetes -name prod-eu-1 \
  -settings '{"kubernetes":{"exclude_namespaces":["kube-system"]}}'
goliash matrix
goliash events -service payments-api -limit 20
goliash service set -name postgres -track minor -pin-major 15
goliash ack -service postgres -kind release -until-version 17.0
```

---

# Collectors
Source: /reference/collectors/

> What each collector reads from Kubernetes, ECS, Lambda, Nomad, Swarm, Docker and Compose, and the permissions it needs.

Collectors only read. They run in the agent, or in the server for targets without an agent.

| Platform | Reads | Needs | Credentials |
| --- | --- | --- | --- |
| Kubernetes | Deployments, StatefulSets, DaemonSets, CronJobs and their running pods (watch) | ClusterRole with `get`, `list`, `watch` | in-cluster service account, else kubeconfig (`kubeconfig_context`) |
| Amazon ECS | clusters, services, running tasks, task definitions | IAM `ecs:List*`, `ecs:Describe*`; `elasticloadbalancing:DescribeTargetGroups` and `elasticloadbalancing:DescribeLoadBalancers` for exposure | default AWS chain; `credentials_ref` names an AWS profile |
| AWS Lambda | functions of a region and their aliases: container image and digest, or the runtime of .zip functions and the packages in their code; tags | IAM `lambda:ListFunctions`, `lambda:GetFunction`, `lambda:ListAliases`; `lambda:ListFunctionUrlConfigs` for exposure; `lambda:GetLayerVersion` for the packages of layers | default AWS chain; `credentials_ref` names an AWS profile |
| Amazon RDS | PostgreSQL, MySQL, MariaDB and Aurora databases of a region: engine version, the versions RDS offers, pending maintenance, public accessibility; tags | IAM `rds:DescribeDBInstances`, `rds:DescribeDBClusters`, `rds:DescribeDBEngineVersions`, `rds:DescribePendingMaintenanceActions` | default AWS chain; `credentials_ref` names an AWS profile |
| Amazon ElastiCache | Redis OSS, Valkey and Memcached caches of a region (replication groups, clusters, serverless): engine version, the versions ElastiCache offers, service updates waiting; tags | IAM `elasticache:Describe*`, `elasticache:ListTagsForResource` | default AWS chain; `credentials_ref` names an AWS profile |
| Amazon EKS | clusters of a region: the Kubernetes version of the control plane and managed node groups, the versions EKS offers, a public API endpoint; tags | IAM `eks:ListClusters`, `eks:DescribeCluster`, `eks:ListNodegroups`, `eks:DescribeNodegroup`, `eks:DescribeClusterVersions` | default AWS chain; `credentials_ref` names an AWS profile |
| Amazon OpenSearch Service | OpenSearch and Elasticsearch domains of a region: engine version, the versions offered, waiting service software update, public endpoint; tags | IAM `es:ListDomainNames`, `es:DescribeDomains`, `es:ListTags`, `es:ListVersions` | default AWS chain; `credentials_ref` names an AWS profile |
| Amazon MSK | provisioned Kafka clusters of a region: Kafka version, the versions MSK offers, public brokers; tags | IAM `kafka:ListClustersV2`, `kafka:ListKafkaVersions` | default AWS chain; `credentials_ref` names an AWS profile |
| Nomad | jobs, job versions, allocations | ACL token with `list-jobs` and `read-job` | `credentials_ref` resolves to the token, else `NOMAD_TOKEN` |
| Docker Swarm | services and running tasks | Docker API `GET` (docker-socket-proxy) | none |
| Docker | running containers, grouped into Compose services; image digests | Docker API `GET` on containers and images | none |
| Compose files | services and images declared in Compose files | an HTTP(S) URL, or for an agent a file in `GOLIASH_COMPOSE_DIRS` | `credentials_ref` → bearer token for the URL |
| Linux server | the operating system (`/etc/os-release`) and installed packages that matter (dpkg, apk) | read access to the host's files: the root mounted at `/host` read-only, or the agent on the host | none |
| Manual | versions people enter on a service page (**Add by hand**), for software Goliash does not collect | nothing: never collected, never stale | none |

## Target settings

Each target has the settings object of its platform:

```json
{"kubernetes": {"kubeconfig_context": "", "include_namespaces": [], "exclude_namespaces": ["kube-system"]}}
{"ecs": {"region": "eu-west-1", "clusters": ["prod"]}}
{"nomad": {"address": "https://nomad.service.consul:4646", "region": "", "namespaces": []}}
{"swarm": {"docker_host": "tcp://socket-proxy:2375"}}
{"docker": {"docker_host": "tcp://socket-proxy:2375", "projects": ["shop"]}}
{"lambda": {"region": "eu-west-1", "name_prefixes": ["shop-"], "alias_environments": {"live": "prod", "canary": "-"}}}
{"rds": {"region": "eu-west-1"}}
{"elasticache": {"region": "eu-west-1"}}
{"eks": {"region": "eu-west-1"}}
{"opensearch": {"region": "eu-west-1"}}
{"msk": {"region": "eu-west-1"}}
{"compose": {"files": ["https://raw.githubusercontent.com/acme/infra/main/compose.yaml"], "project": "shop", "variables": {"TAG": "1.3.0"}}}
```

Kubernetes with Argo CD: when the agent may read `applications.argoproj.io` (the Helm charts grant `get` and `list`),
each workload an Application lists among its resources carries where it is deployed from: the application, its
repository, path or chart, target revision, the commit last synced, and its sync and health status; with
`appprojects.argoproj.io` readable too, whether a sync window keeps it from syncing now, and until when; and whether
it syncs by itself or only by hand (its automated sync policy). With Flux, the
workloads a Kustomization or HelmRelease labelled carry the same from it and its GitRepository or HelmRepository.
Without Argo CD or Flux, or without the permission, nothing changes.

### Exposure

Agents also report how each workload can be reached from outside its cluster or host, so that Updates and
Vulnerabilities put those first (see [Exposed to the outside](../../guide/security/#exposed-to-the-outside)):

| Platform | Exposed when |
| --- | --- |
| Kubernetes | a LoadBalancer service (not marked internal by its cloud's annotation), a NodePort service, an Ingress or a Gateway API HTTPRoute sends to its pods. Needs `list` on `services`, `ingresses.networking.k8s.io` and `httproutes.gateway.networking.k8s.io` (the Helm charts grant it); without it, nothing is exposed. |
| Amazon ECS | the service registers with an internet-facing load balancer (named with its DNS name; internal ones do not count), or its tasks get a public IP. Without `elasticloadbalancing:DescribeTargetGroups` and `DescribeLoadBalancers` every target group counts, marked *internal or not: not known*. |
| AWS Lambda | a function URL anyone may call (auth type `NONE`); needs `lambda:ListFunctionUrlConfigs`. |
| Docker, Compose | a port published on every interface of the host (not on `127.0.0.1`). |
| Amazon RDS | the database (or one of an Aurora cluster's instances) is publicly accessible. |
| Amazon EKS | the cluster's API endpoint is public and open to `0.0.0.0/0`. |
| OpenSearch Service | the domain is outside a VPC (a public endpoint). |
| Amazon MSK | the brokers have public access. |

A `goliash.exposed` label, tag or meta on the workload overrides it: `internet` marks it exposed whatever the agent
saw, `internal` not exposed (an internal load balancer, a VPN-only ingress). On a Kubernetes Service, the label or
annotation `goliash.exposed: internal` leaves that service out.

Nomad: each job is a workload. A periodic or parameterized job is one workload (a cron job) that counts its
running runs, rather than a new workload for every `backup/periodic-…` or `export/dispatch-…` run.
An image written with variables, `image = "ghcr.io/acme/app:${NOMAD_META_version}"`, is read as Nomad would run
it: `${NOMAD_META_<key>}` and `${meta.<key>}` from the job's, group's and task's `meta` (the most specific wins), and
`${NOMAD_JOB_NAME}`, `${NOMAD_GROUP_NAME}`, `${NOMAD_TASK_NAME}` and `${NOMAD_NAMESPACE}`. Variables only a client
knows, such as `${attr.*}` or `${node.*}`, stay as written. Labels (`goliash.app`, `goliash.team`) come from the same
`meta`, and so do `goliash.git.repo`, `goliash.git.path` and `goliash.git.branch`, which name the file in Git that
pins the job's image so that its upgrades open as [pull requests](/guide/argocd/#without-argo-cd-or-flux) (on Docker,
Compose and Swarm they are labels, on ECS the service's tags).

Lambda: each function is a workload, its tags are labels (so `goliash.app` and `goliash.service` work as tags). A
function built from a container image reports that image and digest, checked for new tags like any image. A .zip
function reports its runtime as the matching AWS base image, `python3.12` as `public.ecr.aws/lambda/python:3.12`
and `nodejs20.x` as `public.ecr.aws/lambda/nodejs:20`: Goliash then tells when a newer runtime exists and, from
endoflife.date, when AWS deprecates the one in use. The inbox suggests the function's name as its service, so
functions on the same runtime stay apart.

**The packages in a .zip function's code.** AWS patches the runtime; what can be vulnerable is the code's own
dependencies. Once per function version (its code and layer versions), the agent downloads the code and its layers
from the presigned S3 URL `lambda:GetFunction` and `lambda:GetLayerVersion` give, reads the package manifests in
it (`node_modules/*/package.json`, else `package-lock.json`; Python `*.dist-info/METADATA`; `pom.properties` in
jars, nested ones too; the build information of Go executables such as `bootstrap`; gemspecs; .NET `*.deps.json`)
and sends only the package URLs. Nothing is unpacked to disk or run, the code never leaves the agent, downloads
come only from `*.amazonaws.com` and stop at 300 MB, and a few functions are read per round. Their vulnerabilities
then show like an image's, under the function. Without `lambda:GetLayerVersion` the layers are left out, and the
service page says so; `GOLIASH_LAMBDA_CODE=off` on the agent turns it off. Agents from 1.176 send it.

A function with **aliases** shows once per alias, with the version the alias points to, in the environment named
like the alias: `dev`, `staging` and `prod` aliases fill three columns of the matrix, so a version waiting to be
promoted shows as drift like anywhere else. `alias_environments` maps aliases named otherwise (`live` to `prod`),
and `-` leaves one out (a `canary`); aliases that match no environment stay in the target's. A function without
aliases shows its latest code (`$LATEST`). Without `lambda:ListAliases` the agent reports latest code only and says
so in the target's status.

Empty lists mean everything. `docker_host` takes `tcp://`, `https://` or `unix:///var/run/docker.sock`. The poll
interval is at least 30 seconds and defaults to 300; Kubernetes also sends a snapshot shortly after a change.

## Credentials

The server never sends secrets, only a `credentials_ref` name. The agent resolves it from:

1. the environment variable `GOLIASH_CREDENTIAL_<NAME>`, with the name upper-cased and anything that is not a
   letter or digit replaced by `_`, or
2. the file `$GOLIASH_CREDENTIALS_DIR/<name>` (default `/etc/goliash-agent/credentials`).

## Private registries

Images from registries other than the public ones are checked by the agents. So are private repositories on public
registries (GHCR, Docker Hub, Quay, …): when a registry refuses the server's anonymous check, the repository is
handed to the agents. **Check upstream now** on the service page tries anonymously again, in case it became public.

### Credentials on the server

The quickest way is **Settings → Private registries**: an admin enters the registry host, a username and a
read-only password or token, and the server reads that registry itself for the workspace: new releases, the version
behind moving tags, signatures, attestations, SBOMs and vulnerabilities, with no agent configuration. The panel
lists the registries your services use that only agents can read, each with **Add credentials**; saving tries the
credentials on one of them at once and shows whether the registry accepted them. A token the registry later
refuses is marked *refused* there.

Use a read-only robot account or token: a Harbor or Quay robot, a GitLab deploy token with `read_registry`, a
GitHub token with `read:packages`, a Docker Hub access token, an Artifactory or Nexus read user. The secret is
stored encrypted with the server's secret key (the server refuses to store it without one), bound to its row, never
shown again and never sent to agents; tags read with it are cached for that workspace only. Amazon ECR tokens
expire after 12 hours, so ECR stays with an agent and its IAM role.

Prefer agent credentials when the registry is only reachable from inside your network, or when credentials must not
leave it.

### Credentials on the agent

For each repository the agent looks for credentials for the registry host, in this order:

1. the credential named after the host, for example `GOLIASH_CREDENTIAL_REGISTRY_EXAMPLE_COM` or the file
   `registry.example.com` in the credentials directory;
2. its Docker config: `$DOCKER_CONFIG/config.json`, else `~/.docker/config.json`, as `docker login` writes it.
   Entries kept by a credential helper (`credsStore`, `credHelpers`, as Docker Desktop does) hold no secret and are
   skipped;
3. on Kubernetes, with `GOLIASH_READ_PULL_SECRETS=true` (Helm: `rbac.readPullSecrets`), the image pull secrets that
   running pods of its targets reference. It needs `get` on secrets;
4. for **Google Artifact Registry** (`*-docker.pkg.dev`) and **Container Registry** (`gcr.io`), the service account
   the agent runs as, from the metadata server: GKE Workload Identity or a Compute Engine service account with
   `roles/artifactregistry.reader`;
5. for **Azure Container Registry** (`*.azurecr.io`), AKS Workload Identity: the federated token the pod gets
   (`AZURE_FEDERATED_TOKEN_FILE`, `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`) is exchanged for an ACR token. The identity
   needs the `AcrPull` role;
6. anonymous access.

When a host has several credentials (two pull secrets for different Harbor projects), each is tried in turn.
Credentials never leave the agent.

- **Distribution API registries** (Harbor, GitLab, Artifactory, Nexus, GHCR, Docker Hub, …): `user:password`, or a
  token.
- **Amazon ECR** (`<account>.dkr.ecr.<region>.amazonaws.com`): the agent uses the ECR API with its AWS credentials
  and needs IAM `ecr:ListImages`; with `ecr:GetAuthorizationToken`, `ecr:BatchGetImage` and
  `ecr:GetDownloadUrlForLayer` it also reads the exact version behind moving tags, and the signatures, attestations
  and SBOMs of running images. The default chain is used: IRSA, an ECS task role, an instance profile or the
  environment. A credential, if set, names an AWS profile.

## Linux servers

A server or virtual machine outside containers: the agent reads its operating system from `/etc/os-release` and
its packages from the dpkg (Debian, Ubuntu) or apk (Alpine) database. It only reads files and runs nothing on the
host. RPM-based systems report the operating system only, for now.

```sh
docker run -d --name goliash-agent --restart unless-stopped \
  -v goliash-agent:/data -v /:/host:ro -e GOLIASH_HOST_ROOT=/host \
  -e GOLIASH_SERVER_URL=https://goliash.example.com -e GOLIASH_AGENT_TOKEN=glsh_enroll_… \
  ghcr.io/pipozzz/goliash-agent
```

With `GOLIASH_HOST_ROOT` set, the agent finds the host by itself (named after `/etc/hostname`, identified by
`/etc/machine-id`). **Connect → More ways → Linux server** gives the same command. The Docker agent from **Connect →
Docker** mounts the host's root the same way, so on a Docker host the OS and packages come with the containers.

- **The operating system** is a workload named after it (`ubuntu`, `debian`, `alpine`, `rocky` …) at its version,
  compared with the distribution's image, so its end of life (Ubuntu 20.04, Debian 11 …) shows up like any other.
- **Packages**: servers, runtimes and security-relevant libraries Goliash knows (nginx, Apache, HAProxy, PostgreSQL,
  MySQL, MariaDB, Redis, MongoDB, RabbitMQ, Elasticsearch, Java, Node.js, Python, PHP, Docker, OpenSSL, OpenSSH,
  sudo …), plus the names in `packages`; `all_packages` reports everything. Versions are the upstream part of the
  distribution's (`1.18.0-6ubuntu14.4` → `1.18.0`). Packages with an image of the same software (nginx →
  `docker.io/library/nginx`) get its new releases and end of life; the others (`pkg.goliash/openssl`) are inventory
  only, and their end of life can be named in the service's policy (`"eol": "openssl"`).

```json
{"host": {"root": "/host", "packages": ["keepalived"], "all_packages": false}}
```

---

# Versions and compatibility
Source: /reference/compatibility/

> What Goliash 1.x keeps stable: the REST API, the agent protocol, MCP tools, CLI, configuration, metrics and the database, and how deprecations and support work.

Goliash follows [semantic versioning](https://semver.org/) from 1.0. Within a major version (1.x), what is listed
here keeps working; new things may be added. Anything else, such as the look of the web UI, may change in any
release.

## What stays stable in 1.x

| Interface | Promise |
| --- | --- |
| REST API `/api/v1` ([OpenAPI](/reference/api/)) | No endpoint, parameter or response field is removed or changes meaning. New endpoints, parameters and fields may appear: ignore fields you do not know. |
| Agent protocol `/agent/v1` | A server accepts agents of the same or an older 1.x release (and agents from 0.3 on). Upgrade the server first, then the agents, at your pace. |
| MCP tools | Tool names and arguments stay; new tools and optional arguments may appear. |
| CLI | Commands and flags keep working; output meant for people may change, CSV and JSON output keep their columns and fields. |
| Configuration | Environment variables, Helm chart values and Nomad pack variables keep their meaning and defaults. |
| Metrics | Names and labels of `/metrics` series stay; new series may appear. |
| Webhook payloads | Fields stay; new fields may appear. The signature scheme stays. |
| Database | Migrations only go forward. Any 1.x or 0.x database upgrades by starting a newer server. |

## Deprecations

Something to be removed is first marked deprecated: in the release notes, in these docs, and with a warning in the
server log when it is used. It keeps working for at least one more minor release, and is only removed in the next
major version (2.0).

## Upgrades and support

- Upgrade by starting the newer image or binary; take a [backup](/install/#backups) first. Downgrades are not
  supported: restore the backup instead.
- Every change is tested by upgrading databases written by the first and the latest release, on SQLite and
  PostgreSQL, and by agents of an older release reporting to the new server.
- The latest minor release gets fixes. Security fixes go to the latest minor release; report vulnerabilities
  privately as described in [SECURITY.md](https://github.com/pipozzz/goliash/blob/main/SECURITY.md).
- Releases are signed: see [Verify a release](/security/#verify-a-release).

---

# Configuration
Source: /reference/configuration/

> Environment variables for the Goliash server: database, public URL, e-mail, OIDC, passwords, registries and more.

Settings come from environment variables; most have a matching flag.

## Server (`goliash serve`)

| Variable | Flag | Default | Meaning |
| --- | --- | --- | --- |
| `GOLIASH_DATABASE_URL` | `-database` | `goliash.db` (`/data/goliash.db` in the image) | SQLite file path, or a `postgres://` URL |
| `GOLIASH_PUBLIC_URL` | `-public-url` | `http://localhost:8080` | The address people use; sign-in links and cookies depend on it |
| `GOLIASH_LISTEN` | `-listen` | `:8080` | HTTP listen address |
| `GOLIASH_SECRET_KEY` | | | 32-byte key (base64 or hex) that encrypts channel secrets |
| `GOLIASH_SECRET_KEY_FILE` | | | File with the key, instead of the variable |
| `GOLIASH_GITHUB_TOKEN` | | | Token for GitHub release lookups; raises the rate limit |
| `GOLIASH_GITLAB_URL` | | | A self-hosted GitLab whose releases may be read, e.g. `https://gitlab.example.com` |
| `GOLIASH_GITLAB_TOKEN` | | | Token for the self-hosted GitLab (else for gitlab.com) |
| `GOLIASH_OSV_URL` | | `https://api.osv.dev/v1` | Where the packages of running images' SBOMs are looked up for known vulnerabilities: an OSV API mirror, or `off`. Only package names and versions are sent |
| `GOLIASH_KEV_URL` | | CISA's feed | Where the catalog of known exploited vulnerabilities is read from (a mirror of CISA's JSON), or `off` |
| `GOLIASH_NVD` | | on | `off` stops asking [NVD](https://nvd.nist.gov) for the CVEs of running product versions (PostgreSQL 16.2, nginx 1.25.3…), and for the versions a CVE affects upstream when a distribution has no fix ([Upstream fixes](/guide/security/#upstream-fixes)). Only products, versions and CVE IDs are sent |
| `GOMEMLIMIT` | | ⅔ of the container's memory limit, else of the machine's | Go's soft memory limit: near it the garbage collector works harder instead of letting the heap grow into swap. Set it to choose another, e.g. `300MiB` |
| `GOLIASH_PPROF` | | | An address such as `127.0.0.1:6060` to serve Go's profiles (`/debug/pprof/`) on, for finding what is slow or large on a running server. Keep it on localhost |
| `GOLIASH_NVD_API_KEY` | | | An [NVD API key](https://nvd.nist.gov/developers/request-an-api-key): 50 requests in 30 seconds instead of 5, so new versions and upstream fixes are looked up sooner |
| `GOLIASH_AI_API_KEY` | | | The model provider's API key that turns on [Goli](/guide/goli/), the assistant; also `GOLIASH_AI_PROVIDER` (`anthropic` or `openai`), `GOLIASH_AI_API_KEY_FILE`, `GOLIASH_AI_MODEL` (`claude-opus-5-5`), `GOLIASH_AI_EFFORT`, `GOLIASH_AI_BASE_URL`, `GOLIASH_AI_PER_HOUR`, `GOLIASH_AI_PER_DAY`, `GOLIASH_AI_PLAN_SUMMARY` and `GOLIASH_AI_DIGEST_SUMMARY` |
| `GOLIASH_GOLI_SEASON` | | | What Goli, the mascot, wears: by default it follows the calendar (a pumpkin around Halloween, a Santa hat in December, a scarf in January and February, sunglasses in July and August); `off` never, or `halloween`, `christmas`, `winter`, `summer` always |
| `GOLIASH_SLACK_SIGNING_SECRET` | | | With `GOLIASH_SLACK_GOLI_TOKEN`, turns on Goli's `/goli` command for a Slack app; see [Goli in Slack](/guide/goli/#goli-in-slack). `GOLIASH_SLACK_IN_CHANNEL=true` answers in the channel |
| `GOLIASH_DOCS_ASSISTANT` | | | URL of a product's documentation as one text: Goli answers questions about it for a website at `POST /goli/docs`; see [Goli on your website](/guide/goli/#goli-on-your-website) |
| `GOLIASH_UPDATE_CHECK` | | on | `off` stops the Status page and the `goliash_release` notification from asking `ghcr.io/pipozzz/goliash` for a newer release of Goliash (only its tag list is read). Agents older than the server are flagged on the Status page either way |
| `GOLIASH_ARGOCD_URL` | | | The Argo CD UI, e.g. `https://argocd.example.com`: application names on service pages link to it. See [Argo CD](/guide/argocd/) |
| `GOLIASH_GITOPS_TOKEN` | | | A GitHub token that may write contents and pull requests of the GitOps repositories: upgrades open as pull requests from the Argo CD panel. See [Argo CD](/guide/argocd/#upgrade-with-a-pull-request) |
| `GOLIASH_GITOPS_GITHUB_API` | | `https://api.github.com` | The GitHub API for `GOLIASH_GITOPS_TOKEN`, for GitHub Enterprise |
| `GOLIASH_GITOPS_GITLAB_TOKEN` | | | A GitLab token (`api` scope, Developer on the deploy projects) for merge requests on gitlab.com, or on `GOLIASH_GITLAB_URL` |
| `GOLIASH_GITOPS_GITEA_TOKEN` | | | A Gitea or Forgejo access token (`write:repository`) for pull requests on `GOLIASH_GITEA_URL` |
| `GOLIASH_GITEA_URL` | | | The Gitea or Forgejo instance, e.g. `https://git.example.com` or `https://codeberg.org` |
| `GOLIASH_ALLOWED_REGISTRIES` | | | Comma-separated registries or prefixes images may come from (image hygiene) |
| `GOLIASH_DEBUG` | `-debug` | off | Debug logging |
| `GOLIASH_LOG_FORMAT` | `-log-format` | `text` | `json` for log collectors |
| `GOLIASH_ARGS` | | | The command and flags when the container gets none, e.g. `try -public -reset 1h` on a platform where setting a variable is easier than a command |
| `GOLIASH_LOG_LEVEL` | `-log-level` | `info` | `debug`, `info`, `warn` or `error` |
| `GOLIASH_DRAIN` | `-drain` | `0` | On shutdown, answer `/readyz` with 503 this long before closing |
| `GOLIASH_BACKUP_DIR` | `-backup-dir` | — | SQLite: write a backup here at start and every day |
| `GOLIASH_BACKUP_KEEP` | `-backup-keep` | `7` | Backups kept in the backup directory |
| | `-demo-user` | | A public read-only demo: every visitor is signed in as this existing viewer and every change is refused. `goliash try -public` sets it up |
| | `-collect` | `true` | Collect targets that have no agent in the server itself |
| | `-upstream-interval` | `1h` | How often public registries are checked |
| | `-keep-snapshots` | `20` | Processed snapshots kept per target |
| | `-eol` | `true` | Report end-of-life release cycles from endoflife.date |

| `GOLIASH_BOOTSTRAP` | | | Bootstrap configuration (YAML or JSON), see below |
| `GOLIASH_BOOTSTRAP_FILE` | | | File with the bootstrap configuration, instead of the variable |

Without a secret key, a SQLite installation creates `goliash.key` next to the database. See
[Install](/install/#secrets-at-rest).

### Bootstrap

A bootstrap configuration makes a deployment come up ready, without manual steps: on every start the server
creates the environments, targets and first owner that do not exist yet. Nothing is changed or removed, so edits
made later in the UI are kept.

```yaml
owner: you@example.com            # until they sign in, every start logs a one-time sign-in link
environments:
  - { name: staging, position: 20 }
  - { name: prod, position: 30 }
targets:
  - name: nomad
    environment: prod
    platform: nomad                # kubernetes, ecs, nomad, swarm, docker or compose
    poll_interval_seconds: 60
    # agent: prod-eu               # empty: the server collects the target itself
    settings:
      nomad: { address: "http://10.0.0.5:4646" }
      credentials_ref: nomad       # resolved from GOLIASH_CREDENTIAL_NOMAD
```

Unknown fields are rejected, so a typo stops the server instead of being ignored.

### Web push

| Variable | Meaning |
| --- | --- |
| `GOLIASH_PUSH_SUBJECT` | Who sends, for push services: an `https://` URL or a `mailto:` address. Default: `GOLIASH_PUBLIC_URL` when it is https, else `GOLIASH_SMTP_FROM`. Apple (iPhone, Safari) needs one |

### E-mail

| Variable | Meaning |
| --- | --- |
| `GOLIASH_SMTP_ADDR` | SMTP relay, `host:port` |
| `GOLIASH_SMTP_FROM` | Sender address |
| `GOLIASH_SMTP_USERNAME`, `GOLIASH_SMTP_PASSWORD` | Optional authentication |
| `GOLIASH_SMTP_TLS` | `starttls` (when offered), `tls` (implicit) or `none`; by default TLS on port 465, STARTTLS elsewhere |

With SMTP, sign-in links can be requested by e-mail and e-mail channels work. Without it, an e-mail channel can
carry its own mail server (see [Notifications](/guide/notifications/#e-mail)); that one is used for the channel's
messages only, not for sign-in links.

### Passwords

| Variable | Meaning |
| --- | --- |
| `GOLIASH_RECOVERY_EMAIL` | Locked out without a shell: every start logs a one-time sign-in link for this person. Remove it once you are back in |
| `GOLIASH_SETUP_WINDOW` | While nobody has an account, how long after the start the sign-in page offers to create the first one; default `15m`, `0` keeps the setup link in the log only |
| `GOLIASH_SESSION_TTL` | How long a browser session lasts at most; default `720h` (30 days) |
| `GOLIASH_SESSION_IDLE` | A session unused this long ends; default `336h` (14 days); `0` turns it off |
| `GOLIASH_PASSWORD_LOGIN` | `false` turns password sign-in off (for example when everyone uses OIDC); default on |
| `GOLIASH_TRUST_PROXY` | `true` takes the client address from `X-Forwarded-For`. Set it only behind a reverse proxy, which must set the header |

The client address goes into the session list, the audit log and the sign-in limits: 8 failed passwords for an
address, or 30 from one client, pause sign-in for 15 minutes.

### OIDC sign-in

Owners can set this up in **Settings → Single sign-on** instead; the variables below take precedence when set.

| Variable | Meaning |
| --- | --- |
| `GOLIASH_OIDC_ISSUER` | Issuer URL, e.g. `https://accounts.google.com` |
| `GOLIASH_OIDC_CLIENT_ID`, `GOLIASH_OIDC_CLIENT_SECRET` | Client credentials |
| `GOLIASH_OIDC_NAME` | Button label, e.g. `Google` |
| `GOLIASH_OIDC_DOMAINS` | Comma-separated e-mail domains whose people get a viewer account on first sign-in |
| `GOLIASH_GROUP_ROLES` | Roles from the provider's groups, e.g. `goliash-admins=admin, sre=member, sre-pay=member@payments, everyone=viewer` (see below) |
| `GOLIASH_OIDC_GROUPS_CLAIM` | The ID token claim with the groups; default `groups` |
| `GOLIASH_OIDC_SCOPES` | More scopes to ask for, e.g. `groups` for providers that send groups only then |

Redirect URI: `<GOLIASH_PUBLIC_URL>/auth/oidc/callback`.

#### Roles from groups

With `GOLIASH_GROUP_ROLES` the identity provider decides who may sign in and with which role, at every sign-in:

- `group=admin` makes its people organization admins (every workspace).
- `group=member` or `group=viewer` gives that role in every workspace. `group=member@payments` gives it in one
  workspace, by slug or name.
- Someone in several groups gets the highest role each grants, per workspace.
- Someone in none of the groups is refused, also with an existing account. Someone new in one gets an account
  without an invitation (`GOLIASH_OIDC_DOMAINS` is then not needed).
- Leaving a group takes its role away at the next sign-in. Owners keep being owners.

Roles changed on the settings page are set again at the next sign-in, so change them at the provider. Groups match
by name, or as a path (Keycloak's `/sre`); Entra ID sends group object IDs unless the app registration emits names,
so either use the IDs in the mapping or configure *groups claim → sAMAccountName / cloud-only group names*. Okta and
Authentik send groups when asked for the `groups` scope (`GOLIASH_OIDC_SCOPES=groups`).

### SCIM provisioning

| Variable | Meaning |
| --- | --- |
| `GOLIASH_SCIM_TOKEN` | Turns SCIM 2.0 on at `<GOLIASH_PUBLIC_URL>/scim/v2`, with this bearer token (at least 32 characters: `openssl rand -hex 32`) |

With SCIM the identity provider (Entra ID, Okta, Authentik, Keycloak with a SCIM plugin, JumpCloud) keeps the people
of Goliash in step with its own:

- **New people** are created when the app is assigned to them. They can then sign in with OIDC (or a magic link).
- **Deactivation** (`active: false`, when someone leaves or is unassigned) ends their sessions and the apps they
  let in at once, and refuses every way to sign in. Reactivating restores them. Settings → People marks them
  *deactivated*. Deleting removes them.
- **Pushed groups** decide roles as `GOLIASH_GROUP_ROLES` maps them, the same mapping OIDC sign-in uses: someone
  in no mapped group has an account but no workspace. Without `GOLIASH_GROUP_ROLES`, new people are viewers of the
  first workspace and roles are set in Goliash.
- **Owners** are never changed by groups, and the last active owner cannot be deactivated or deleted.

Users are found by `userName` (the e-mail address), and the provider's ID is kept as `externalId`. PATCH works in
the forms Entra ID and Okta send (`"active": "False"` included). Every change is in the audit log as
`scim.user.*` or `scim.group.*`.

Setting it up in Entra ID: *Enterprise applications → the Goliash app → Provisioning → Automatic*, tenant URL
`https://goliash.example.com/scim/v2`, secret token `GOLIASH_SCIM_TOKEN`. In Okta: *Provisioning → SCIM 2.0*, base URL
the same, authentication *HTTP Header* with the token, and turn on *Push Groups*.

## CLI

Every command other than `serve` takes `-database` (`GOLIASH_DATABASE_URL`) and `-workspace`
(`GOLIASH_WORKSPACE`, default the first workspace). Flags go after the command: `goliash matrix -workspace acme`.

## Agent (`goliash-agent`)

| Variable | Flag | Default | Meaning |
| --- | --- | --- | --- |
| `GOLIASH_SERVER_URL` | `-server` | | Server URL |
| `GOLIASH_AGENT_TOKEN` | | | The enrollment code (`glsh_enroll_…`) or the agent's token (`glsh_agent_…`) |
| `GOLIASH_AGENT_TOKEN_FILE` | | | File with the code or token, instead of the variable |
| `GOLIASH_ENROLL_CODE` | | | The enrollment code, if you prefer to keep it apart from tokens |
| `GOLIASH_AGENT_NAME` | | found where it runs | Name of an enrolling agent and of the target it finds |
| `GOLIASH_AGENT_ID` | | found where it runs | What tells this installation apart when it enrolls with a code for many agents |
| `GOLIASH_AWS` | | on | `off` stops the agent from asking EC2 instance metadata for its region and role |
| `GOLIASH_REDISCOVER` | | `15m` | How often the agent looks again for what it could collect, and enrolls or registers again when it finds more; `off` looks only at start |
| `GOLIASH_AWS_REGIONS` | | | More AWS regions an enrolling agent adds ECS clusters and Lambda functions of, comma-separated |
| `GOLIASH_ECS_CLUSTERS` | | every cluster | Only these ECS clusters, by name, comma-separated |
| `GOLIASH_ECS_EXCLUDE` | | | ECS clusters, by name, an enrolling agent leaves out (the CloudFormation template sets its own cluster) |
| `GOLIASH_LAMBDA_ALIASES` | | | Lambda aliases named otherwise than environments, e.g. `live=prod,canary=-` (`-` leaves one out) |
| `GOLIASH_LAMBDA` | | on | `off` stops an enrolling agent from adding its AWS region's Lambda functions |
| `GOLIASH_LAMBDA_CODE` | | on | `off` stops the agent from downloading .zip Lambda functions' code and layers to read the packages in them ([Collectors](/reference/collectors/)) |
| `GOLIASH_TARGETS` | | | Targets an enrolling agent declares, as a JSON array of [agent protocol targets](/reference/agent-protocol/) without `id` |
| `GOLIASH_DATA_DIR` | `-data-dir` | `data` (`/data` in the image) | Buffered snapshots while the server is unreachable |
| `GOLIASH_CREDENTIALS_DIR` | | `/etc/goliash-agent/credentials` | Directory of credential files |
| `GOLIASH_CREDENTIAL_<NAME>` | | | A credential; see [Collectors](/reference/collectors/#credentials) |
| `GOLIASH_COMPOSE_DIRS` | | | Comma-separated directories `compose` targets may read files from |
| `GOLIASH_DEBUG` | `-debug` | off | Debug logging |
| `GOLIASH_LOG_FORMAT` | `-log-format` | `text` | `json` for log collectors |

Everything else (targets, intervals, which registries to check) the agent reads from the server. With an
[enrollment code](/install/#enrollment-codes) the agent adds the targets it finds itself.

---

# AI assistants (MCP)
Source: /reference/mcp/

> Ask Claude, Cursor and other AI assistants what runs where through the Goliash MCP server.

Goliash speaks the [Model Context Protocol](https://modelcontextprotocol.io), so an AI assistant such as Claude or
Cursor can answer "which version of payments runs in prod, and is it behind upstream?", "what changed in prod in the
last two hours?", "what reaches end of life soon?" or "is Log4Shell running anywhere?" from live data.

Every tool calls the REST API with an API token, so the assistant sees and may do exactly what the token allows.
Create one for it; a viewer token only reads, a member token may also acknowledge:

```sh
goliash token create -name assistant -expires 90d            # reads
goliash token create -name assistant -role member -expires 90d  # also acknowledges
```

## Tools

| Tool | What it answers |
| --- | --- |
| `search` | Search everything at once: services by name, image, version, workload or GitOps application; vulnerabilities; packages; pull requests; history; notifications. The first tool for "where does openssl run?" |
| `matrix` | What runs where, filtered by service or environment, now or at a past time |
| `service` | One service: versions per environment, newest upstream release, open drift |
| `drifts` | What needs attention: env, upstream, inconsistent, declared, eol |
| `updates` | What to upgrade, most urgent first: running → target version, release notes and why; filter by owner, environment or urgency |
| `release_notes` | What changed between two versions of a service: the release notes in between, from its GitHub or GitLab releases |
| `changes` | What changed: recently (default: last 24 hours), or in a window of the past (`since` and `before` as times) |
| `promotions` | What waits for the next environment, with the releases it brings, the vulnerabilities it fixes and brings, and the packages it brings under demanding licenses |
| `rollouts` | Rollouts that do not finish: workloads down, stuck with two versions side by side, or short of replicas |
| `gitops` | Is Git what runs? Argo CD applications and Flux Kustomizations: repository, path, commit, sync and health; for a degraded one, the version change just before it and so the version to roll back to; sync windows and manual sync |
| `pull_requests` | Pull requests Goliash opened to upgrade, promote or roll back: open, merged or closed, by whom, their checks, whether the version runs now and how long after merging it began to; for a merged one not deployed, why it waits |
| `delivery` | Deploys per environment and lead times, last 30 days |
| `inventory` | Every running container with image and digest, now or at a past time, with image size and whether its base is current |
| `images` | Why did an image grow? One service's images: size against its budget, base and how far behind it is, evidence, 90 days of sizes, and the layers the last deploy added and removed with the Dockerfile step of each |
| `hygiene` | Moving tags, tags pushed again, untrusted registries, images built on an outdated base |
| `licenses` | Licenses of the packages in running images, most demanding first (AGPL, BUSL, GPL, unknown…), with where each runs |
| `vulnerabilities` | Is a CVE running? Known vulnerabilities of running images, exploited ones first, with the services and the version that fixes each |
| `posture` | The security posture: exposure days (median, p90, 30+, 90+), end of life, accepted risk, blind spots (with why each service's newest release is unknown) and every exposure |
| `notifications` | What Goliash sent: alerts, digests and upgrade plans, each once, newest first (default: 7 days) |
| `status` | Whether the workspace is in a change freeze (`change_freeze`), and Goliash's own health: agents reporting, releases checked, vulnerability data current, notifications going out; whether an answer can be trusted |
| `acknowledge` | Quiet notifications until a version or a time (needs a member token; the only tool that writes) |

## Remote (HTTP)

Every Goliash server serves MCP at `/mcp` (Streamable HTTP). Apps sign in with **OAuth**, the way the MCP
specification describes, so no token is copied anywhere:

```sh
claude mcp add --transport http goliash https://goliash.example.com/mcp
```

The first time, `/mcp` → **Authenticate** in Claude Code (Claude Desktop, Cursor and VS Code do the same) opens
Goliash in the browser: sign in as usual, pick the workspace and **Allow**. The app then reads Goliash as you:

- **Read only**, whatever your role; tools that change something (`acknowledge`) are refused.
- What you can see in that workspace, and only while you are a member of it.
- Access tokens last an hour and the app renews them; after 30 days without use it has to ask again.
- **Your account → Apps with access** lists every app with the workspace and when it last read, and **Revoke** ends
  it at once.
- Admins see every app in the workspace, and whose it is, under **Users → Apps with access**, and can revoke any.

For Cursor, in `.cursor/mcp.json`, the URL is enough:

```json
{ "mcpServers": { "goliash": { "url": "https://goliash.example.com/mcp" } } }
```

### Try it on the demo

The [live demo](https://demo.goliash.dev) takes AI assistants too, with no account:

```sh
claude mcp add --transport http goliash-demo https://demo.goliash.dev/mcp
```

Then `/mcp` → **Authenticate**, **Allow** on the page that opens, and ask: *"What should we upgrade first in
prod, and why?"*, *"What changed in staging this week?"*, *"Which services run past their end of life?"* The
assistant reads the demo's example data as its read-only viewer.

### With an API token

Scripts, CI jobs and apps without OAuth send a viewer (or member) API token instead; **Integrations → AI
assistants** makes one and fills in the command:

```sh
claude mcp add --transport http goliash https://goliash.example.com/mcp \
  --header "Authorization: Bearer glsh_api_…"
```

### How the sign-in works

Goliash is both the MCP server and its OAuth 2.1 authorization server:

| | |
| --- | --- |
| Discovery | `/mcp` answers 401 with `resource_metadata`; `/.well-known/oauth-protected-resource` (RFC 9728) names the server, `/.well-known/oauth-authorization-server` (RFC 8414) its endpoints |
| App identity | Client ID Metadata Documents (the app's `client_id` is an https URL Goliash fetches, from the public internet only), or Dynamic Client Registration (RFC 7591) for older apps |
| Sign-in | `/oauth/authorize`: your usual sign-in (password, passkey, single sign-on, two-factor), then a consent page; authorization codes with PKCE (S256) only, ten minutes, single use; `iss` in the answer (RFC 9207) |
| Tokens | `/oauth/token`: bound to `https://…/mcp` (`resource`, RFC 8707), access one hour, refresh 30 days and rotated on every use; `/oauth/revoke` (RFC 7009) |
| Redirects | https, or http on a loopback address (any port, RFC 8252); anything else is refused, and an unregistered one is never followed |

`GOLIASH_PUBLIC_URL` should be set: it is the issuer apps check, and the address in every link.

## Local (stdio)

`goliash mcp` runs an MCP server on stdin and stdout that talks to your Goliash server, for apps that only start
local MCP servers. After `goliash login https://goliash.example.com` it needs nothing else (`"args": ["mcp"]`); with
an API token instead, for Claude Desktop, in `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "goliash": {
      "command": "goliash",
      "args": ["mcp"],
      "env": { "GOLIASH_URL": "https://goliash.example.com", "GOLIASH_TOKEN": "glsh_api_…" }
    }
  }
}
```

---

# Security
Source: /security/

> How Goliash stays read-only, protects tokens, sessions and secrets, and how to verify signed releases.

## Read-only by design

- **Collectors only read.** The Kubernetes ClusterRole allows `get`, `list` and `watch`; the ECS role
  `ecs:List*`, `ecs:Describe*` and `ecr:ListImages`; Nomad needs `list-jobs` and `read-job`; Docker and Swarm go through a
  docker-socket-proxy that allows only `GET`. Nothing in Goliash creates, changes or deletes anything in your
  infrastructure.
- **The agent only connects out.** It sends snapshots to the server over HTTPS. The server never connects to the
  agent, so no inbound port is needed in your network.
- **Credentials stay with the agent.** The server sends only the name of a credential; the agent resolves it from
  its own environment or files.

## Tokens and sessions

- Agent tokens (`glsh_agent_…`) and API tokens (`glsh_api_…`) are shown once and stored as SHA-256 hashes. They
  carry a checksum, so secret scanners can recognize them. An agent token can only send data and read that agent's
  configuration.
- Passwords are hashed with argon2id (19 MiB, 2 passes) and must have at least 12 characters. Failed sign-ins are
  limited per address and per client, answer the same whether or not the account exists, and take the same time.
- Two-factor sign-in with TOTP codes (RFC 6238) is available to everyone and also guards sign-in links; codes
  cannot be replayed, recovery codes are single-use and stored hashed, the secret is encrypted at rest.
- Passkeys (WebAuthn) sign in without a password and count as two factors: bound to the server's domain, with user
  verification on the device. Only public keys are stored; each challenge is answered once within five minutes.
- Owners can require two-factor sign-in for everyone who does not use single sign-on or a passkey.
- Sessions end after 30 days, or 14 days unused (both configurable). Request bodies are capped at 1 MB outside the
  agent protocol, which has its own limits.
- A server without accounts offers to create the first (owner) account on its sign-in page for 15 minutes after
  the start, like other self-hosted tools; whoever opens it first owns the organization. Start it where only you
  can reach it, or set `GOLIASH_SETUP_WINDOW=0` to keep the one-time setup link in the server log only.
- Sign-in links are single-use and short-lived. Sessions are HttpOnly cookies, marked Secure when
  `GOLIASH_PUBLIC_URL` uses `https://`. Browsers may not send state-changing requests from other origins.
- OIDC sign-in uses PKCE, state and nonce.
- Every response carries a strict Content-Security-Policy (only the server's own scripts, no inline script, no
  framing), `X-Frame-Options: DENY`, `X-Content-Type-Options: nosniff`, `Referrer-Policy: same-origin` and, when
  `GOLIASH_PUBLIC_URL` uses `https://`, `Strict-Transport-Security`.

## Secrets at rest

Notification channel secrets and private registry credentials are encrypted with AES-256-GCM and bound to their
row; registry credentials are never stored without a key, never shown again after saving and never sent to agents.
See
[Install](/install/#secrets-at-rest). Webhook deliveries can be signed with HMAC-SHA256; see
[Notifications](/guide/notifications/#webhook-payload).

## The assistant

[Goli](/guide/goli/) is off unless the server has a model configured (Anthropic, or an OpenAI-compatible API). When on, it reads through the REST API with
the asking person's session, so it sees what they may see, and it has no tool that changes anything. What it looks
up goes to that model's API with the question; credentials and secrets never do. The log records who asked and which
tools were used, not the question.

## Audit log

Every change to configuration, tokens and roles, and every sign-in, is recorded with who made it. Secrets never
appear in the log.

## Verify a release

Images and the release checksums are signed with [cosign](https://github.com/sigstore/cosign) in the release
workflow, without long-lived keys:

```sh
cosign verify ghcr.io/pipozzz/goliash:0.1.0 \
  --certificate-identity-regexp '^https://github.com/pipozzz/goliash/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
```

For binaries, download `checksums.txt` and `checksums.txt.sigstore.json` from the release, then:

```sh
cosign verify-blob checksums.txt --bundle checksums.txt.sigstore.json \
  --certificate-identity-regexp '^https://github.com/pipozzz/goliash/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
sha256sum --ignore-missing -c checksums.txt        # macOS: shasum -a 256 --ignore-missing -c checksums.txt
```

Each image and archive comes with an SPDX SBOM.

## Reporting a vulnerability

Please report vulnerabilities privately through
[GitHub security advisories](https://github.com/pipozzz/goliash/security/advisories/new), not in public issues.

---

# REST API (OpenAPI 3)
Source: /reference/api/

```yaml
# Copyright 2026 The Goliash Authors
# SPDX-License-Identifier: Apache-2.0
openapi: 3.0.3
info:
  title: Goliash API
  version: 1.0.0
  description: |
    Read what runs where, on which version, and acknowledge releases and drift.

    - Authenticate with `Authorization: Bearer glsh_api_…` (an API token from **Users → API tokens**) or a browser
      session. A token belongs to one workspace; every response is about that workspace.
    - `GET` endpoints need the viewer role, `POST /api/v1/acks` the member role. An API token has the role
      chosen when it was created: viewer (the default) or member.
    - Errors use `application/problem+json` (RFC 9457).
    - Prometheus metrics for the same data are at `GET /metrics` (text exposition, same authentication).
  license:
    name: Apache-2.0
    url: https://www.apache.org/licenses/LICENSE-2.0
servers:
  - url: https://goliash.example.com
security:
  - apiToken: []
  - session: []

paths:
  /api/v1/matrix:
    get:
      operationId: getMatrix
      summary: The service × environment matrix
      description: |
        Every service with the versions running in each environment, open drift and the newest upstream version.
        With `at`, the matrix as it was then, without drift and upstream data.
      parameters:
        - $ref: '#/components/parameters/At'
      responses:
        "200":
          description: The matrix.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Matrix'
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/services:
    get:
      operationId: listServices
      summary: Services
      responses:
        "200":
          description: Services by name.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Service'
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/services/{name}/release-notes:
    get:
      operationId: getReleaseNotes
      summary: What changed between two versions of a service
      description: >-
        The release notes of every release after `from` up to and including `to`, newest first, read from the
        GitHub or GitLab releases the service's policy, its image's source label or the catalog names. At most
        25 releases and 80 KiB of notes; `omitted` counts the rest.
      parameters:
        - { name: name, in: path, required: true, schema: { type: string } }
        - { name: from, in: query, description: "The version running now; notes start after it. Empty: only to's notes.", schema: { type: string, example: "1.27.2" } }
        - { name: to, in: query, description: "The version to move to; default the newest upstream release.", schema: { type: string } }
      responses:
        "200":
          description: The notes.
          content:
            application/json:
              schema:
                type: object
                required: [service, to, releases]
                properties:
                  service: { type: string }
                  from: { type: string }
                  to: { type: string }
                  source: { type: string, description: "github:owner/repo or gitlab:host/project; absent when none is known", example: "github:nginx/nginx" }
                  changelog: { type: string, description: "The changelog of to, when its notes live outside a forge." }
                  omitted: { type: integer }
                  releases:
                    type: array
                    items:
                      type: object
                      required: [version, notes]
                      properties:
                        version: { type: string }
                        url: { type: string }
                        published_at: { type: string, format: date-time }
                        notes: { type: string, description: "Markdown, as the project wrote it." }
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
        "404": { $ref: '#/components/responses/NotFound' }

  /api/v1/services/{name}/images:
    get:
      operationId: getServiceImages
      summary: What a service's images are made of
      description: >-
        Per environment, the running main image's size against the service's size budget (`max_size_mb`), its base
        and whether that base has a newer build, and its supply-chain evidence; every image that ran in the last 90
        days with its size; and what the last deploy (production first) changed in the layers, biggest first, with
        the Dockerfile step that made each. Sizes, bases and layers appear once the image's registry was read.
      parameters:
        - { name: name, in: path, required: true, schema: { type: string } }
      responses:
        "200":
          description: The images.
          content:
            application/json:
              schema:
                type: object
                required: [service, images, history]
                properties:
                  service: { type: string }
                  budget_bytes: { type: integer, format: int64, description: "The size budget; absent without one." }
                  images:
                    type: array
                    items:
                      type: object
                      required: [environment, image, digest]
                      properties:
                        environment: { type: string }
                        image: { type: string }
                        version: { type: string }
                        digest: { type: string }
                        size_bytes: { type: integer, format: int64, description: "What pulling it downloads: compressed layers and config." }
                        over_budget: { type: boolean }
                        layers: { type: integer }
                        base_image: { type: string, example: "docker.io/library/node:22-alpine" }
                        base_status: { type: string, enum: [current, outdated] }
                        base_behind_days: { type: integer, description: "Days the base tag has had a newer build than the image is built on." }
                        base_chain:
                          type: array
                          description: "Down from base_image: what each image is built on (from its base label or BuildKit provenance), current or outdated against its tag now. An outdated link makes base_status outdated."
                          items:
                            type: object
                            required: [image, base, status]
                            properties:
                              image: { type: string }
                              base: { type: string }
                              status: { type: string, enum: [current, outdated] }
                              how: { type: string, enum: [label, provenance] }
                        rebuild_first: { type: string, description: "The image of the chain built on an older base, when not this one: rebuild it first, then this image." }
                        evidence:
                          type: object
                          required: [signed, sbom, provenance]
                          properties:
                            signed: { type: boolean }
                            sbom: { type: boolean }
                            provenance: { type: boolean }
                            signers:
                              type: array
                              description: Who signed it, as its keyless signatures' certificates name them (not verified).
                              items:
                                type: object
                                required: [identity]
                                properties:
                                  identity: { type: string, example: "https://github.com/acme/api/.github/workflows/release.yml@refs/heads/main" }
                                  issuer: { type: string, example: "https://token.actions.githubusercontent.com" }
                  history:
                    type: array
                    items:
                      type: object
                      required: [environment, digest, since]
                      properties:
                        environment: { type: string }
                        version: { type: string }
                        digest: { type: string }
                        size_bytes: { type: integer, format: int64 }
                        since: { type: string, format: date-time }
                        until: { type: string, format: date-time, description: "Absent while it still runs." }
                  last_change:
                    type: object
                    required: [environment, from, to, before_bytes, after_bytes, added, removed, added_count, removed_count]
                    properties:
                      environment: { type: string }
                      from: { type: string }
                      to: { type: string }
                      before_bytes: { type: integer, format: int64 }
                      after_bytes: { type: integer, format: int64 }
                      added: { type: array, items: { $ref: '#/components/schemas/ImageLayer' } }
                      removed: { type: array, items: { $ref: '#/components/schemas/ImageLayer' } }
                      added_count: { type: integer, description: "All added layers; added lists the six biggest." }
                      removed_count: { type: integer }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
        "404": { $ref: '#/components/responses/NotFound' }

  /api/v1/licenses:
    get:
      operationId: listLicenses
      summary: Licenses of the packages in running images
      description: >-
        Every package the SBOMs of running images list, with the license the SBOM names and its class, most
        demanding first: network copyleft (AGPL, SSPL), source-available (BUSL, Elastic), strong copyleft (GPL), weak
        copyleft (LGPL, MPL), unknown, permissive. An OR in a license expression counts as its least demanding
        choice, an AND as its most demanding part. Operating system packages (deb, apk, rpm) are marked `os`.
      parameters:
        - { name: format, in: query, description: "csv: one row per package and where it runs, for a license review.", schema: { type: string, enum: [json, csv], default: json } }
        - { name: environment, in: query, description: Only this environment., schema: { type: string } }
        - name: class
          in: query
          description: "Only this class, or attention: network copyleft, source-available, strong copyleft and unknown."
          schema: { type: string, enum: [attention, network-copyleft, source-available, strong-copyleft, weak-copyleft, unknown, permissive] }
        - { name: os, in: query, description: "false leaves out operating system packages.", schema: { type: boolean } }
        - { name: q, in: query, description: "A package name, purl, license or service.", schema: { type: string, example: "AGPL" } }
      responses:
        "200":
          description: The packages.
          content:
            text/csv:
              schema: { type: string }
            application/json:
              schema:
                type: object
                required: [images, images_with_sbom, counts, os_counts, complete, packages]
                properties:
                  images: { type: integer, description: Running images with a digest. }
                  images_with_sbom: { type: integer }
                  counts: { type: object, additionalProperties: { type: integer }, description: "Application packages per class." }
                  os_counts: { type: object, additionalProperties: { type: integer }, description: "Operating system packages per class." }
                  complete: { type: boolean, description: "false while SBOMs read before licenses were kept wait to be read again." }
                  packages:
                    type: array
                    items:
                      type: object
                      required: [purl, name, class, os, running_in]
                      properties:
                        purl: { type: string, example: "pkg:maven/com.itextpdf/itextpdf@5.5.13.3" }
                        name: { type: string }
                        version: { type: string }
                        license: { type: string, description: "As the SBOM names it, an SPDX expression; absent when it names none.", example: "AGPL-3.0-only" }
                        class: { type: string, enum: [network-copyleft, source-available, strong-copyleft, weak-copyleft, unknown, permissive] }
                        os: { type: boolean }
                        running_in:
                          type: array
                          items:
                            type: object
                            required: [service, environment]
                            properties:
                              service: { type: string }
                              environment: { type: string }
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
        "404": { $ref: '#/components/responses/NotFound' }

  /api/v1/notifications/sent:
    get:
      operationId: listSentNotifications
      summary: What Goliash sent
      description: >-
        Alerts, digests and upgrade plans sent by any rule, newest first, each message once however many channels it
        went to: what the bell lists. At most 100 messages.
      parameters:
        - { name: since, in: query, description: "How far back: a duration like 24h or 30d, or an RFC 3339 time. Default: 7 days.", schema: { type: string } }
      responses:
        "200":
          description: The messages.
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  required: [sent_at, title, mode, items]
                  properties:
                    sent_at: { type: string, format: date-time }
                    title: { type: string }
                    mode: { type: string, enum: [instant, daily, weekly], description: "The rule's delivery: at once or in a digest." }
                    items:
                      type: array
                      items:
                        type: object
                        required: [type, at, text]
                        properties:
                          type: { type: string, description: "An event type, update (upgrade plan), agent_stale or agent_back." }
                          service: { type: string }
                          app: { type: string }
                          owner: { type: string }
                          environment: { type: string }
                          target: { type: string }
                          from: { type: string }
                          to: { type: string }
                          note: { type: string }
                          at: { type: string, format: date-time }
                          text: { type: string }
                          url: { type: string }
        "400": { $ref: '#/components/responses/BadRequest' }
  /api/v1/summary:
    get:
      operationId: getSummary
      summary: Goliash in a dozen numbers
      description: >-
        What the tiles show, for widgets (iPhone with Scriptable, Windows 11 Widgets, wall screens): services,
        drifting, updates waiting and how many have exploited vulnerabilities, production's exposure and end of life,
        stale targets.
      responses:
        "200":
          description: The numbers.
          content:
            application/json:
              schema:
                type: object
                required: [workspace, production, services, drifting, updates, exploited, behind, median_exposure_days, behind_90_days, end_of_life_passed, stale_targets, generated_at, generated]
                properties:
                  workspace: { type: string }
                  production: { type: string, description: "The last environment." }
                  services: { type: integer }
                  drifting: { type: integer, description: "Services with open drift." }
                  updates: { type: integer, description: "Updates waiting, not put off by an acknowledgement." }
                  exploited: { type: integer, description: "Of them, how many run an image with a vulnerability exploited in the wild." }
                  behind: { type: integer, description: "Production services behind a newer release, not counting accepted risk." }
                  median_exposure_days: { type: integer }
                  behind_90_days: { type: integer }
                  end_of_life_passed: { type: integer }
                  stale_targets: { type: integer }
                  generated_at: { type: string, format: date-time }
                  generated: { type: string, description: "generated_at as text, e.g. Oct 14, 13:05 UTC." }
  /api/v1/posture:
    get:
      operationId: getPosture
      summary: The security posture
      description: >-
        How long production has run behind available releases, what is past or near its end of life, accepted risk
        and blind spots, with every exposure and the version that fixes it: the Security page as data, for risk
        registers, GRC tools and AI assistants.
      parameters:
        - { name: all, in: query, description: "true: exposures in every environment, not only production", schema: { type: boolean } }
      responses:
        "200":
          description: The posture.
          content:
            application/json:
              schema:
                type: object
                required: [production, services_running, median_exposure_days, p90_exposure_days, behind_30_days, behind_90_days, end_of_life_passed, end_of_life_soon, accepted_risk, stale_targets, no_known_upstream, exposures, history]
                properties:
                  production: { type: string, description: "The last environment, where exposure counts most." }
                  services_running: { type: integer }
                  median_exposure_days: { type: integer, description: "Over production exposures whose release dates are known." }
                  p90_exposure_days: { type: integer }
                  behind_30_days: { type: integer }
                  behind_90_days: { type: integer }
                  end_of_life_passed: { type: integer }
                  end_of_life_soon: { type: integer }
                  accepted_risk: { type: integer, description: "Production exposures quieted by an acknowledgement." }
                  stale_targets: { type: array, items: { type: string } }
                  no_known_upstream: { type: array, items: { type: string }, description: "Services whose exposure cannot be measured." }
                  unknown_upstream_why:
                    type: array
                    description: "Why each of those services' newest release is unknown, and what would let Goliash tell."
                    items:
                      type: object
                      required: [service, short, reason]
                      properties:
                        service: { type: string }
                        repository: { type: string }
                        short: { type: string, example: not a version }
                        reason: { type: string, example: "It runs the tag \"latest\", which is not a version: there is nothing to compare the releases with." }
                        fix: { type: string }
                  outdated_base: { type: array, items: { type: string }, description: "Production services whose image is built on an older build of its base than the base's tag has now: rebuild with --pull." }
                  history:
                    type: array
                    description: "Production's posture once a week over the last 90 days, oldest first: is patching getting faster?"
                    items:
                      type: object
                      required: [day, behind, median_exposure_days, p90_exposure_days, behind_30_days, behind_90_days, end_of_life_passed, accepted_risk]
                      properties:
                        day: { type: string, format: date }
                        behind: { type: integer, description: "Production services behind a newer release, not counting accepted risk." }
                        median_exposure_days: { type: integer }
                        p90_exposure_days: { type: integer }
                        behind_30_days: { type: integer }
                        behind_90_days: { type: integer }
                        end_of_life_passed: { type: integer }
                        accepted_risk: { type: integer }
                  exposures:
                    type: array
                    items:
                      type: object
                      required: [service, environment, running, exposed_days, since, since_known, why, accepted]
                      properties:
                        service: { type: string }
                        app: { type: string }
                        environment: { type: string }
                        running: { type: string }
                        fix: { type: string, description: "The version to move to." }
                        release_notes: { type: string }
                        exposed_days: { type: integer }
                        since: { type: string, format: date-time }
                        since_known: { type: boolean, description: "false: counted from when Goliash first saw the newer release, a lower bound." }
                        why: { type: string }
                        end_of_life: { type: string }
                        accepted: { type: boolean }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/environments:
    get:
      operationId: listEnvironments
      summary: Environments in promotion order
      responses:
        "200":
          description: Environments, lowest position first.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Environment'
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/targets:
    get:
      operationId: listTargets
      summary: Targets (clusters, regions, hosts) and their collectors
      responses:
        "200":
          description: Targets.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Target'
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/events:
    get:
      operationId: listEvents
      summary: History of deploys, releases and drift, newest first
      parameters:
        - name: service
          in: query
          description: Service name.
          schema: { type: string }
        - name: environment
          in: query
          description: Environment name.
          schema: { type: string }
        - name: type
          in: query
          description: Comma-separated event types.
          schema: { type: string, example: "version_changed,new_release" }
        - name: before
          in: query
          description: Only events before this time (RFC 3339), for paging with the last event's `at`.
          schema: { type: string, format: date-time }
        - name: since
          in: query
          description: Only events from this time on, as a duration back from now (`2h`, `30m`, `7d`) or RFC 3339.
          schema: { type: string, example: 2h }
        - name: limit
          in: query
          schema: { type: integer, minimum: 1, maximum: 1000, default: 100 }
      responses:
        "200":
          description: Events, newest first.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Event'
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
        "404": { $ref: '#/components/responses/NotFound' }

  /api/v1/drifts:
    get:
      operationId: listDrifts
      summary: Open drift, oldest first
      responses:
        "200":
          description: Open drifts.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Drift'
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/promotions:
    get:
      operationId: listPromotions
      summary: Versions waiting to be promoted, longest waiting first
      description: |
        An environment that runs something newer than the environment after it, with the releases a promotion
        would bring.
      responses:
        "200":
          description: Pending promotions.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Promotion'
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/rollouts:
    get:
      operationId: listRollouts
      summary: Rollouts that do not finish
      description: >-
        Workloads whose replicas are not where they should be, worst first: `down` (none of the replicas it asks for
        runs), `stuck` (two versions side by side for more than 30 minutes) and `short` (fewer replicas than it asks
        for). Only workloads whose platform reports the replicas it asks for (Kubernetes, ECS, Nomad, Swarm, Compose).
      responses:
        "200":
          description: The workloads.
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  required: [kind, service, environment, target, workload, running, desired, since, versions]
                  properties:
                    kind: { type: string, enum: [down, stuck, short] }
                    service: { type: string }
                    environment: { type: string }
                    target: { type: string }
                    workload: { type: string, example: shop/payments-api }
                    running: { type: integer }
                    desired: { type: integer }
                    since: { type: string, format: date-time, description: "Since when it has been so, as far as Goliash can tell." }
                    versions:
                      type: array
                      description: The versions running, newest first.
                      items:
                        type: object
                        required: [version, running, since]
                        properties:
                          version: { type: string }
                          running: { type: integer }
                          since: { type: string, format: date-time }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/gitops:
    get:
      operationId: listGitOps
      summary: GitOps applications
      description: >-
        The Argo CD applications and Flux Kustomizations that deploy the workspace's services, those that need
        attention first: where they deploy from, the commit last synced, sync and health, and since when. `trouble` is
        `out of sync`, `degraded` or `missing` once the state has lasted 10 minutes, else empty. For one degraded or
        missing, `after_change` is the version change on its cluster in the day before that began: the likeliest
        cause, and `from` the version to roll back to. Empty without Argo CD or Flux.
      responses:
        "200":
          description: The applications, per workload they deploy.
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  required: [tool, app, service, environment, target, workload, repo, path, target_revision, revision, sync, health, since, trouble]
                  properties:
                    tool: { type: string, enum: [argocd, flux] }
                    app: { type: string, example: checkout-prod }
                    service: { type: string }
                    environment: { type: string }
                    target: { type: string }
                    workload: { type: string, example: shop/checkout }
                    repo: { type: string, example: https://github.com/acme/deploy.git }
                    path: { type: string, description: "The path in the repository, or the Helm chart." }
                    target_revision: { type: string, example: main }
                    revision: { type: string, description: "The commit or chart version last synced." }
                    sync: { type: string, example: OutOfSync }
                    health: { type: string, example: Healthy }
                    since: { type: string, format: date-time, description: "Since when sync and health are what they are." }
                    trouble: { type: string, enum: ["", "out of sync", "degraded", "missing"] }
                    version: { type: string, description: "The version (image tag) of the workload's main container.", example: 25.0.6 }
                    prev_revision: { type: string, description: "The commit synced before revision, once it changed." }
                    compare_url: { type: string, description: "The forge's page of what changed in Git from prev_revision to revision." }
                    sync_blocked: { type: string, description: "Why the application does not sync now: Argo CD's sync windows, or suspended in Flux; absent otherwise." }
                    manual_sync: { type: boolean, description: "Argo CD syncs it only by hand (no automated sync policy): a merged change waits for that." }
                    sync_blocked_until: { type: string, format: date-time, description: "When that sync window closes, or the next allow window opens." }
                    after_change:
                      type: object
                      description: For a degraded or missing application, the version change just before it.
                      required: [from, to, at]
                      properties:
                        from: { type: string, example: 25.0.1 }
                        to: { type: string, example: 25.0.6 }
                        at: { type: string, format: date-time }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/search:
    get:
      operationId: search
      summary: Search everything
      description: >-
        Every word of q must appear. Services by name, owner, image, version, workload or GitOps application; known
        vulnerabilities of running images (CVE, advisory, package, service); packages with their licenses; the pull
        requests Goliash opened; the last 90 days of history and the last 30 of notifications. Each section counts
        its matches and lists the first `limit`.
      parameters:
        - { name: q, in: query, required: true, schema: { type: string }, example: openssl }
        - { name: limit, in: query, schema: { type: integer, default: 10, maximum: 50 }, description: "Matches listed per section." }
      responses:
        "200":
          description: The sections, every kind present even when empty.
          content:
            application/json:
              schema:
                type: object
                required: [query, total, sections]
                properties:
                  query: { type: string }
                  total: { type: integer }
                  sections:
                    type: array
                    items:
                      type: object
                      required: [kind, title, total, hits]
                      properties:
                        kind: { type: string, enum: [services, vulnerabilities, packages, pull_requests, history, notifications] }
                        title: { type: string }
                        total: { type: integer }
                        more: { type: string, description: "The page in Goliash listing all of them, filtered." }
                        hits:
                          type: array
                          items:
                            type: object
                            required: [label, url]
                            properties:
                              label: { type: string, example: CVE-2024-6119 }
                              detail: { type: string }
                              url: { type: string, description: "A path in Goliash, or a pull request's address when external." }
                              badge: { type: string, example: exploited }
                              external: { type: boolean }
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
  /api/v1/pull-requests:
    get:
      operationId: listPullRequests
      summary: Pull requests Goliash opened
      description: >-
        The pull requests Goliash opened in GitOps repositories to upgrade, promote or roll back a service, newest
        first: open, merged or closed as the forge last said (asked every 30 minutes), who opened them (`auto-pr` for
        a version policy), when they were merged, whether the version they pin runs on that workload now and how long
        it took, and for a merged one that does not run yet why it waits.
      responses:
        "200":
          description: The pull requests.
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  required: [service, environment, target, from, to, url, state, opened_by, opened_at, running]
                  properties:
                    service: { type: string, example: checkout }
                    environment: { type: string, example: prod }
                    target: { type: string, example: k8s-prod }
                    from: { type: string, example: 2.3.1 }
                    to: { type: string, example: 2.4.0 }
                    url: { type: string, example: https://github.com/acme/deploy/pull/7 }
                    state: { type: string, enum: [open, merged, closed] }
                    opened_by: { type: string, example: auto-pr }
                    opened_at: { type: string, format: date-time }
                    running: { type: boolean, description: "The version it pins runs on that workload now." }
                    checks: { type: string, enum: [passing, failing, pending], description: "Its checks (CI), while open." }
                    merged_at: { type: string, format: date-time, description: "When it was merged, as the forge says." }
                    waiting: { type: string, description: "Merged, and the version does not run: why it waits, when Goliash can tell.", example: Argo CD syncs it by hand }
                    to_running_seconds: { type: integer, description: "From opening it to the version running there." }
                    merge_to_running_seconds: { type: integer, description: "From merging it to the version running there." }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/delivery:
    get:
      operationId: getDelivery
      summary: Delivery statistics per service
      description: |
        Versions that arrived in each environment within the window, and the median time a version took from one
        environment to the next, read from what ran when.
      parameters:
        - name: window
          in: query
          schema: { type: string, default: 30d, example: 7d }
      responses:
        "200":
          description: Statistics per service.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Delivery'
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/inventory:
    get:
      operationId: getInventory
      summary: Every running container, for audits
      description: |
        Every container that runs now (or ran at `at`), sidecars included, with image, tag and digest.
        `format=cyclonedx` returns a CycloneDX 1.6 BOM instead: one component per image (an OCI purl and its
        digest, where it runs as `goliash:` properties) with the packages of its SBOM nested inside, where the
        registry holds one, ready for Dependency-Track or an auditor.
      parameters:
        - $ref: '#/components/parameters/At'
        - name: format
          in: query
          schema: { type: string, enum: [json, csv, cyclonedx], default: json }
        - name: environment
          in: query
          description: Only what runs in this environment, e.g. one Dependency-Track project per environment.
          schema: { type: string }
      responses:
        "200":
          description: The inventory.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/InventoryItem'
            text/csv:
              schema: { type: string }
            application/vnd.cyclonedx+json:
              schema: { type: object, description: "A CycloneDX 1.6 JSON document." }
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
        "404": { $ref: '#/components/responses/NotFound' }

  /api/v1/status:
    get:
      operationId: getStatus
      summary: Goliash's own health
      description: |
        Whether Goliash itself works and its data is fresh, as Settings → Status shows it: one check per part
        (agents, targets, upstream releases, private registries, vulnerability data, notifications, Goliash's
        version) and the worst state. For monitors and assistants deciding whether an answer can be trusted.
      responses:
        "200":
          description: The checks.
          content:
            application/json:
              schema:
                type: object
                required: [checks, state, checked_at]
                properties:
                  state: { type: string, enum: [ok, warn, bad, off] }
                  checked_at: { type: string, format: date-time }
                  change_freeze:
                    type: object
                    description: The change freeze the workspace is in now, absent outside one.
                    required: [until]
                    properties:
                      until: { type: string, format: date-time }
                      note: { type: string, example: Black Friday sale }
                  checks:
                    type: array
                    items:
                      type: object
                      required: [title, state, summary]
                      properties:
                        title: { type: string }
                        state: { type: string, enum: [ok, warn, bad, off] }
                        summary: { type: string }
                        details: { type: array, items: { type: string } }
                        link: { type: string, description: "Where in the UI to look, relative to the server." }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/updates:
    get:
      operationId: getUpdates
      summary: What to upgrade, most urgent first
      description: |
        One item per service (and application, for a service compared per application) and environment with open
        end-of-life, upstream or environment drift: what runs, the version to move to and why. Ordered by urgency
        (an exploited vulnerability in the running image, end of life, end of life soon, major, minor, behind the previous
        environment, patch), production first.
        Items an acknowledgement quiets are included with `acknowledged: true`.
      responses:
        "200":
          description: Updates, most urgent first.
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  required: [service, environment, running, urgency, since, acknowledged]
                  properties:
                    service: { type: string }
                    owner: { type: string }
                    app: { type: string }
                    environment: { type: string }
                    running: { type: string }
                    target: { type: string, description: "The version to move to." }
                    release_notes: { type: string, format: uri }
                    jump: { type: string, enum: [patch, minor, major] }
                    behind: { type: string, description: "The previous environment it is behind." }
                    eol: { type: string, format: date }
                    urgency: { type: string, enum: ["exploited vulnerability", "security fix", "end of life", "end of life soon", major, minor, "behind previous env", patch] }
                    security_fix: { type: string, description: "What the releases up to target say they fix: CVE, GHSA or GO identifiers, or \"security\"." }
                    breaking:
                      type: array
                      items: { type: string }
                      description: "What the releases up to target say breaks, quoted from their notes, oldest first: \"2.0.0: Removed support for PostgreSQL 12\"."
                    due: { type: string, format: date, description: "The remediation deadline the workspace set for its most pressing reason (Security page), when it set one." }
                    due_for: { type: string, enum: ["exploited vulnerability", "security fix", "end of life"] }
                    days_left: { type: integer, description: "Days until due: negative once past." }
                    exploited:
                      type: array
                      items: { type: string }
                      description: Vulnerabilities in the running image that CISA lists as exploited in the wild, from its SBOM.
                    critical_vulnerabilities: { type: integer, description: "Other critical vulnerabilities in the running image." }
                    exposed:
                      type: array
                      items: { type: string }
                      description: How the running version can be reached from outside its cluster or host; exposed updates come first within an urgency.
                    maintenance:
                      type: array
                      items: { type: string }
                      description: Patches the platform has waiting for a managed service (RDS pending maintenance, ElastiCache service updates, OpenSearch service software).
                    vulnerabilities:
                      allOf: [{ $ref: '#/components/schemas/VulnDelta' }]
                      description: >-
                        What the upgrade does to known vulnerabilities, when the target version already runs somewhere:
                        comparing the SBOMs of its image and the running one. Absent unless both have an SBOM.
                    since: { type: string, format: date-time }
                    acknowledged: { type: boolean }
        "401": { $ref: '#/components/responses/Unauthorized' }

  /api/v1/hygiene:
    get:
      operationId: getHygiene
      summary: Image hygiene findings
      description: |
        Running images with a moving tag (latest, stable …), a tag pushed again (same tag, different digests),
        a registry outside `GOLIASH_ALLOWED_REGISTRIES`, or no known digest (info).
      responses:
        "200":
          description: Findings, warnings first.
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  required: [kind, severity, image, detail, where]
                  properties:
                    kind: { type: string, enum: [moving-tag, retagged, untrusted-registry, unpinned] }
                    severity: { type: string, enum: [warning, info] }
                    image: { type: string }
                    detail: { type: string }
                    where:
                      type: array
                      items: { type: string }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/vulnerabilities:
    get:
      operationId: getVulnerabilities
      summary: Known vulnerabilities of running images
      description: |
        Is a vulnerability running? From the SBOMs of running images and OSV: one finding per CVE (else advisory),
        with every advisory naming it, whether CISA lists it as exploited in the wild, and the packages and
        services it is in with the version that fixes each. `complete` is false while advisories or packages are
        still being looked up: an empty answer then does not mean "not running". Only images with an SBOM are
        covered (`images_with_sbom` of `images`).
      parameters:
        - { name: format, in: query, description: "csv: one row per vulnerability and affected image, with CISA due dates and fixes.", schema: { type: string, enum: [json, csv], default: json } }
        - { name: q, in: query, schema: { type: string }, description: 'A CVE or advisory ID, package or service; part of it matches' }
        - { name: environment, in: query, schema: { type: string }, description: Only this environment }
        - { name: exploited, in: query, schema: { type: boolean }, description: Only vulnerabilities exploited in the wild (CISA KEV) }
        - { name: exposed, in: query, schema: { type: boolean }, description: Only those in images reachable from outside their cluster or host }
      responses:
        "200":
          description: Findings, exploited first, then those in exposed images, then by severity.
          content:
            text/csv:
              schema: { type: string }
            application/json:
              schema:
                type: object
                required: [images, images_with_sbom, packages, complete, findings]
                properties:
                  images: { type: integer }
                  images_with_sbom: { type: integer }
                  packages: { type: integer }
                  complete: { type: boolean }
                  findings:
                    type: array
                    items:
                      type: object
                      required: [id, advisories, aliases, fixable, affected]
                      properties:
                        id: { type: string, example: CVE-2021-44228 }
                        advisories: { type: array, items: { type: string } }
                        aliases: { type: array, items: { type: string } }
                        summary: { type: string }
                        severity: { type: string, example: critical }
                        exploited:
                          type: object
                          required: [ransomware]
                          properties:
                            due_date: { type: string }
                            date_added: { type: string }
                            ransomware: { type: boolean }
                            past_due: { type: boolean, description: "It still runs after the due date CISA set for fixing it (BOD 22-01)." }
                        fixable: { type: boolean }
                        running_since: { type: string, format: date-time, description: "Since when it has run: the earliest of the affected images." }
                        affected:
                          type: array
                          items:
                            type: object
                            required: [service, environment, image, package]
                            properties:
                              service: { type: string }
                              environment: { type: string }
                              image: { type: string }
                              package: { type: string, example: log4j-core 2.14.1 }
                              fix: { type: string, example: 2.15.0 }
                              since: { type: string, format: date-time, description: "Since when the image has run in the environment." }
                              exposed:
                                type: array
                                items: { type: string }
                                description: How the image can be reached from outside its cluster or host, as the agent read it.
                                example: ["ingress shop.example.com"]
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
        "404": { $ref: '#/components/responses/NotFound' }

  /api/v1/acks:
    post:
      operationId: createAck
      summary: Acknowledge a release or a drift
      description: |
        Silences notifications for a service (optionally in one environment) until a version ships or a time passes.
        When the workspace asks for approval of accepted risks, one made with a member's token needs a `reason`,
        lasts at most 90 days, and quiets nothing until an admin approves it on the Security page (`pending`).
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/AckRequest'
      responses:
        "201":
          description: Acknowledged.
          content:
            application/json:
              schema:
                type: object
                required: [id, pending]
                properties:
                  id: { type: string }
                  pending: { type: boolean, description: "Waits for an admin's approval." }
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }
        "404": { $ref: '#/components/responses/NotFound' }

  /api/v1/check:
    post:
      operationId: checkImage
      summary: Check an image before it is deployed
      description: |
        For CI (goliash check): the vulnerabilities of the image's packages (from the SBOM handed in, its SBOM
        attestation, or its package database, looked up on OSV and CISA KEV), its signature against the service's
        policy, and the end of life of its release cycle. The answer is 200 whether the image passes or not.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [image]
              properties:
                image: { type: string, example: ghcr.io/acme/payments-api:1.6.0 }
                service: { type: string, description: The service whose policy applies; default the one running this repository. }
                sbom: { type: object, description: The SPDX or CycloneDX JSON document the build made. }
                fail_on:
                  type: array
                  items: { type: string, enum: [exploited, eol, signer, unsigned, critical, no-sbom, unreadable, none] }
                  description: The rules that fail the check; default exploited, eol, signer, unsigned. none fails on nothing.
      responses:
        "200":
          description: The verdict.
          content:
            application/json:
              schema:
                type: object
                required: [image, passed, findings, packages, sbom_source, signed]
                properties:
                  image: { type: string }
                  digest: { type: string }
                  service: { type: string }
                  passed: { type: boolean }
                  findings:
                    type: array
                    items:
                      type: object
                      required: [rule, blocking, detail]
                      properties:
                        rule: { type: string }
                        blocking: { type: boolean }
                        detail: { type: string }
                  packages: { type: integer }
                  sbom_source: { type: string, enum: [request, attestation, packages, none] }
                  signed: { type: boolean }
                  signers: { type: array, items: { type: string } }
                  exploited: { type: array, items: { type: string } }
                  critical: { type: array, items: { type: string } }
                  eol: { type: string, description: "The end of life of its release cycle, when known." }
        "400": { $ref: '#/components/responses/BadRequest' }
        "401": { $ref: '#/components/responses/Unauthorized' }
        "403": { $ref: '#/components/responses/Forbidden' }

  /api/v1/openapi.yaml:
    get:
      operationId: getSpec
      summary: This specification
      security: []
      responses:
        "200":
          description: OpenAPI 3.0 document.
          content:
            application/yaml:
              schema: { type: string }

components:
  securitySchemes:
    apiToken:
      type: http
      scheme: bearer
      description: An API token (`glsh_api_…`).
    session:
      type: apiKey
      in: cookie
      name: goliash_session

  parameters:
    At:
      name: at
      in: query
      description: A point in the past, RFC 3339 or `2026-09-12T14:00` (UTC). History is kept for 400 days.
      schema: { type: string, example: "2026-09-12T14:00:00Z" }

  responses:
    BadRequest:
      description: The request is invalid.
      content:
        application/problem+json:
          schema: { $ref: '#/components/schemas/Problem' }
    Unauthorized:
      description: No valid token or session.
      content:
        application/problem+json:
          schema: { $ref: '#/components/schemas/Problem' }
    Forbidden:
      description: The role is not enough.
      content:
        application/problem+json:
          schema: { $ref: '#/components/schemas/Problem' }
    NotFound:
      description: Unknown service or environment.
      content:
        application/problem+json:
          schema: { $ref: '#/components/schemas/Problem' }

  schemas:
    ImageLayer:
      type: object
      required: [size_bytes]
      properties:
        size_bytes: { type: integer, format: int64 }
        command: { type: string, description: "The build step that made it, from the image config's history.", example: "RUN apt-get install -y chromium" }
    Environment:
      type: object
      required: [id, name, position]
      properties:
        id: { type: string }
        name: { type: string, example: prod }
        position:
          type: integer
          description: Promotion order; lower comes first (dev 10, staging 20, prod 30).

    Service:
      type: object
      required: [id, name, owner, kind, upstream, version_policy]
      properties:
        id: { type: string }
        name: { type: string, example: payments-api }
        owner: { type: string, example: team-payments }
        kind: { type: string, enum: [own, third_party] }
        upstream:
          type: string
          description: Image repository releases are read from; empty means the image the service runs.
        version_policy:
          $ref: '#/components/schemas/VersionPolicy'

    VersionPolicy:
      type: object
      description: The service's own policy; empty means the catalog's or the defaults.
      properties:
        track: { type: string, enum: [patch, minor, major] }
        pin_major: { type: integer }
        tag_filter: { type: string }
        prerelease: { type: boolean }
        drift_alert_after:
          type: object
          additionalProperties: { type: string }
          example: { env: 168h }
        github: { type: string, example: acme/payments }
        gitlab: { type: string, example: gitlab.com/acme/payments }
        eol: { type: string, example: postgresql, description: 'endoflife.date product; "none" turns end-of-life drift off' }
        github_tag_prefix: { type: string }
        changelog: { type: string }

    Target:
      type: object
      required: [id, name, platform, environment, collected_by]
      properties:
        id: { type: string }
        name: { type: string, example: prod-eu-1 }
        platform: { type: string, enum: [kubernetes, ecs, nomad, swarm, docker] }
        environment: { type: string }
        collected_by:
          type: string
          description: The agent's ID, or `server` when the server collects the target itself.
        last_snapshot_at: { type: string, format: date-time }
        collector_status: { type: string, enum: [ok, degraded, failing] }
        collector_error: { type: string }

    Version:
      type: object
      required: [tag, running, targets]
      properties:
        tag: { type: string, example: 1.27.3 }
        digest: { type: string }
        running:
          type: integer
          description: Running replicas.
        targets:
          type: array
          items: { type: string }

    DriftDetail:
      type: object
      required: [running, other]
      properties:
        running: { type: string }
        other:
          type: string
          description: The newer version, upstream's or the lower environment's.
        other_in:
          type: string
          description: The environment running `other` (env drift).
        jump: { type: string, enum: [patch, minor, major] }
        targets:
          type: object
          description: For inconsistent drift, the version per target.
          additionalProperties: { type: string }
        eol:
          type: string
          format: date
          description: For eol drift, when the release cycle (`other`) stops being supported.

    Drift:
      type: object
      required: [service, environment, kind, since, days_open, detail]
      properties:
        service: { type: string }
        environment: { type: string }
        app:
          type: string
          description: |
            The application, when the service is used by several and its versions are compared within each
            (one postgres image, many databases). Absent when the service drifts as a whole.
        kind:
          type: string
          enum: [env, upstream, inconsistent, declared, eol]
          description: |
            `env`: behind a lower environment. `upstream`: behind the newest release by the tracked jump.
            `inconsistent`: targets of one environment run different versions. `declared`: what runs differs
            from what the Compose files in Git declare. `eol`: the running release cycle reaches its end of life
            within 60 days or has reached it (endoflife.date).
        since: { type: string, format: date-time }
        days_open: { type: number }
        detail: { $ref: '#/components/schemas/DriftDetail' }

    Cell:
      type: object
      required: [environment, versions, drifts]
      properties:
        environment: { type: string }
        versions:
          type: array
          items: { $ref: '#/components/schemas/Version' }
        declared:
          type: array
          description: What Compose files (in Git) declare, when it is reported besides what runs.
          items: { $ref: '#/components/schemas/Version' }
        from_declared:
          type: boolean
          description: Nothing reports what runs; `versions` come from Compose files.
        drifts:
          type: array
          items: { $ref: '#/components/schemas/Drift' }

    MatrixRow:
      type: object
      required: [service, cells]
      properties:
        service: { type: string }
        owner: { type: string }
        upstream: { type: string }
        latest:
          type: string
          description: The newest upstream version the policy accepts.
        cells:
          type: array
          description: One per environment, in the order of `environments`.
          items: { $ref: '#/components/schemas/Cell' }

    Matrix:
      type: object
      required: [environments, services, unmapped]
      properties:
        environments:
          type: array
          items: { $ref: '#/components/schemas/Environment' }
        services:
          type: array
          items: { $ref: '#/components/schemas/MatrixRow' }
        unmapped:
          type: integer
          description: Running workloads not mapped to a service yet (the inbox).

    Event:
      type: object
      required: [id, type, source, at]
      properties:
        id: { type: string }
        type:
          type: string
          example: version_changed
          description: deployed, version_changed, removed, new_release, drift_detected, drift_resolved or agent_stale.
        service: { type: string }
        app: { type: string, description: "The application of a drift event, when the service is compared per application." }
        environment: { type: string }
        target: { type: string }
        from: { type: string }
        to: { type: string }
        note: { type: string, description: "retag (same version, new digest) or rollback (a version_changed to an older version); for drift, its kind." }
        source: { type: string, description: "Where the event came from, e.g. snapshot or poll." }
        at: { type: string, format: date-time }

    Promotion:
      type: object
      required: [service, from, to, version, running, since, releases]
      properties:
        service: { type: string }
        app: { type: string, description: "The application, for a service compared per application." }
        from: { type: string, description: The environment running the newer version. }
        to: { type: string, description: The environment behind. }
        version: { type: string, description: Waiting in `from`. }
        running: { type: string, description: Running in `to`. }
        since: { type: string, format: date-time }
        releases:
          type: array
          description: Releases after `running` up to `version`, newest first.
          items:
            type: object
            required: [version]
            properties:
              version: { type: string }
              published_at: { type: string, format: date-time }
              release_notes: { type: string }
        vulnerabilities:
          allOf: [{ $ref: '#/components/schemas/VulnDelta' }]
          description: >-
            What promoting does to the known vulnerabilities of `to`, comparing the SBOMs of the two versions' images on
            OSV: those it fixes and those it brings, exploited ones first. Absent unless both images have an SBOM.
        licenses:
          type: array
          description: >-
            Application packages the promoted version brings, or relicenses, under network copyleft, source-available
            or strong copyleft terms, from the SBOMs of both images; not legal advice. Absent when there are none.
          items:
            type: object
            required: [package, version, license, class]
            properties:
              package: { type: string, example: github.com/hashicorp/vault }
              version: { type: string, example: 1.15.0 }
              license: { type: string, example: BUSL-1.1 }
              class: { type: string, enum: [network-copyleft, source-available, strong-copyleft] }
              was: { type: string, description: "Its license in the running version, when it had the package.", example: MPL-2.0 }

    VulnDelta:
      type: object
      required: [fixed, new]
      properties:
        fixed: { type: array, items: { $ref: '#/components/schemas/VulnRef' } }
        new: { type: array, items: { $ref: '#/components/schemas/VulnRef' } }

    VulnRef:
      type: object
      required: [id, exploited]
      properties:
        id: { type: string, example: CVE-2021-44228 }
        severity: { type: string, example: CRITICAL }
        exploited: { type: boolean, description: CISA lists it as exploited in the wild. }

    Delivery:
      type: object
      required: [service, environments, lead_times]
      properties:
        service: { type: string }
        environments:
          type: array
          items:
            type: object
            required: [environment, deploys]
            properties:
              environment: { type: string }
              deploys: { type: integer, description: Versions that arrived within the window. }
              rollbacks: { type: integer, description: "Changes back to an older version within the window: with deploys, a change failure rate." }
              last_deploy: { type: string, format: date-time }
        lead_times:
          type: array
          items:
            type: object
            required: [from, to, median_seconds, samples]
            properties:
              from: { type: string }
              to: { type: string }
              median_seconds: { type: number }
              samples: { type: integer }

    InventoryItem:
      type: object
      required: [environment, target, platform, workload, kind, container, image, running, first_seen]
      properties:
        environment: { type: string }
        target: { type: string }
        platform: { type: string }
        service: { type: string }
        namespace: { type: string }
        workload: { type: string }
        kind: { type: string }
        container: { type: string }
        image: { type: string }
        tag: { type: string }
        digest: { type: string }
        running: { type: integer }
        first_seen: { type: string, format: date-time }
        size_bytes: { type: integer, format: int64, description: "What pulling the image downloads (compressed layers and config), when read." }
        base: { type: string, description: "The FROM image the image was compared with, e.g. docker.io/library/node:22-alpine." }
        base_status: { type: string, enum: [current, outdated], description: "outdated: built on an older build of its base than the base's tag has now (rebuild with --pull)." }

    AckRequest:
      type: object
      required: [service, kind]
      properties:
        service: { type: string }
        environment:
          type: string
          description: Empty acknowledges the service in every environment.
        kind: { type: string, enum: [release, drift] }
        until_version:
          type: string
          description: Quiet until this version runs. Set this or `until`.
        until:
          type: string
          format: date-time
          description: Quiet until this time.
        reason:
          type: string
          description: Why the risk is accepted; required when it needs approval.

    Problem:
      type: object
      description: RFC 9457 problem details.
      required: [title, status]
      properties:
        type: { type: string, default: about:blank }
        title: { type: string }
        status: { type: integer }
        detail: { type: string }
        instance: { type: string }
```