# MSP Guide -- Running ErezAgent for Multiple Clients

**Version: v4.94.5**

This guide is for Managed Service Providers running ErezAgent for multiple clients, or as a
white-label service desk. It's organized by topic, not by release -- every capability is described
as it works today, with a small `(since vX.Y.Z)` tag where the version matters. For installation
and upgrade mechanics, see `docs/DEPLOYMENT_GUIDE.md`; for day-to-day configuration and governance
that applies to any deployment (single-tenant or multi-client), see `ADMIN_GUIDE.md` -- this
guide focuses specifically on what changes when you're running one or more client businesses
instead of your own.

---

## 1. Why ErezAgent for MSPs

ErezAgent is an AI Company Operating System, not a chatbot -- it runs missions, agents, and
governed actions against a real business's data, with every high-risk action gated behind your
approval. For an MSP, that makes it a genuine multi-client operating layer rather than a per-seat
tool:

- **AI service desk** -- automated Tier 1/2 ticket routing, response drafting, and escalation, per
  client.
- **Client health monitoring** -- a cross-client portfolio view: health scores, SLA compliance (as entered),
  open/critical tickets, ARR, and renewal risk, all in one place.
- **Proactive intelligence** -- anomaly detection, capacity warnings, and CVE alerts surfaced to
  the right team, scoped to the right client.
- **Automation with a human in the loop** -- agents handle repetitive work (provisioning checks,
  backup verification, monthly reporting) but nothing destructive or client-facing executes without
  your approval.
- **One governance model across every client** -- the same approval engine, risk tiers, and audit
  trail apply whether you're running one business or twenty.

Everything in this guide assumes you've already read `ADMIN_GUIDE.md` for the mechanics of
roles, connectors, backups, and security policy -- those apply identically per tenant; this guide
covers what's specifically different about running several of them.

---

## 2. Deployment Models for MSPs

Two deployment models both give you real, database-enforced client isolation -- pick based on
your compliance posture and operational preference.

### Option A -- One instance per client (recommended for strict compliance)

```
client-a.yourcompany.com -> ErezAgent stack A (own DB, own Redis, own API keys, own containers)
client-b.yourcompany.com -> ErezAgent stack B
```

Each client gets a fully separate deployment: no shared database, no shared Redis, no shared
process. This is the right choice when a client's contract or regulatory environment requires
physically separate infrastructure, not just logical separation. Follow `docs/DEPLOYMENT_GUIDE.md`
for each client's install; there's no special "MSP" install flag to set -- it's the same Web
Installer Wizard or shell installer you'd use for a single-tenant deployment, run once per client.
With the client's consent, each installation can report its status to your platform-owner
instance automatically; see "Provider Fleet page" in section 10.

### Option B -- Shared instance, multiple tenants

```
shared.yourcompany.com -> one ErezAgent deployment, N client tenants, each with its own
                            database rows -- never its own database
```

A single deployment serves every client as a separate tenant. Each tenant's data is isolated at
the PostgreSQL row level -- see section 3 for exactly how that guarantee works. Owner/operator/
observer roles are built-in RBAC per user; a client's own admin user only ever sees their own
tenant's data, board, and dashboards. This model is operationally lighter (one set of containers to
patch, monitor, and back up) and is what most MSPs running more than a handful of clients choose.

A permanent automated check fails the build if any tenant-scoped table is added without row-level
security, so isolation does not need to be re-verified per release. (The live
`GET /api/tenants/rls-status` diagnostic is owner-only and is available on the platform owner's own
instance only; other installs return 403 for it.)

---

## 3. Tenant Isolation -- The Real Guarantee

This is the mechanism every other section of this guide relies on. Read it once here; nothing else
in this guide re-explains it.

Every tenant-scoped table is protected by PostgreSQL Row-Level Security, enforced through a
dedicated, non-superuser database role (`erezagent_app`) rather than the database's bootstrap
superuser account -- this matters because PostgreSQL row-level security policies do not apply to a
superuser connection by design, so ErezAgent's application traffic deliberately never connects as
one. `TenantContextMiddleware` extracts the caller's tenant from their JWT on every request and
sets it as a transaction-local Postgres setting; every RLS policy checks that setting before
allowing a row to be read or written. A permanent, automated CI check fails the build if a new
table is ever added without tenant scoping, so this guarantee holds as the platform grows -- it is
not something you need to re-verify release over release.

Background workers and migrations run with no tenant setting, which is the deliberate, audited
convention this system uses for "operate across all tenants" (running a scheduled job for every
client, applying a schema migration) -- never a bypass of a real per-request boundary.

**What this means in practice for your client fleet:**

- One client's users, missions, agents, connectors, contracts, revenue data, support cases, and
  every other tenant-scoped record are database-enforced invisible to another client's session --
  not filtered at the application layer, where a missed `WHERE` clause could leak a row.
- A client's own admin (operator or owner role on their own tenant) only ever sees their own
  tenant's records -- the database itself refuses any other row.
- This is the same guarantee whether you run Option A (one stack per client) or Option B (shared
  instance, many tenants) -- Option A adds physical separation on top of it; it doesn't provide
  isolation that Option B lacks.
- This automatically covers newer per-client data too, with no MSP-specific configuration --
  for example, each client's GTM motions, segments, offers, and commercial-intelligence view
  (`since v4.75.0`) are as database-enforced isolated as their revenue, support, and product data.
- **A second, narrower isolation boundary sits inside each client's tenant (since v4.84.0):** a
  client's own venture staff -- employees they invite scoped to one venture only, not a full
  admin account -- authenticate through a distinct JWT scope that's structurally rejected by
  every route except the small set explicitly built for it, and every read/write is checked
  against the caller's live venture membership on top of the tenant-level RLS boundary above.
  You don't configure or manage this -- it's a within-tenant guarantee your client's own owner
  controls from their venture's Staff panel, invisible to you unless they ask for help.
- **A third, narrower surface for a client's own end customers (since v4.85.0):** if a client
  turns on a Business Record type's Customer Portal, their customer reaches only their own
  record, at a link scoped to that one venture and that one record -- re-verified on every
  request, never trusted from the link itself. Same as the staff boundary above, this is
  configured entirely by your client's own owner from their record type's builder; nothing for
  you to set up, and no change to the tenant-level isolation guarantee it sits inside.
- **Each client has its own AI workforce settings (since v4.88.0):** the core agents (CEO, CFO and the rest) are shared
  templates, but every tenant keeps its own enabled flag, status, daily token budget, default model, prompt override and
  department for them. A client's owner changes only their own tenant's values (owner-only); another client, and the
  template itself, are never affected, and the tenant isolation above applies to these settings like any other
  tenant-scoped record.
- **Business models are per tenant and per venture (since v4.88.0):** what a client uploads to Build My Business
  (documents, extracted text, the proposed model, change history, record migrations) is stored under that client's tenant and
  venture and is invisible to every other client. Uploaded files are deleted automatically after the client's retention
  period (`TWIN_SOURCE_RETENTION_DAYS`, 90 days by default). AI reading of a client's documents is off until that client's
  owner grants consent in Settings > Data & Privacy, and the consent is per tenant, so granting it for one client never
  applies to another.

**Shared/Option B installs (since v4.76.0):** the owner-only unhandled-API-error log
(`/api/admin/api-errors...`) is now scoped per tenant: a client's owner sees only their own
tenant's errors, plus errors with no tenant context yet. No action is needed on your end; it
applies automatically on upgrade.

**Install-wide administration on shared installs (v4.89.0):** a few surfaces act on the whole install
rather than one client: Backup Center (backups, restore, downloads, the disaster-recovery secrets
escrow), feature flags and the kill switch, secret rotation, installing the install-wide license,
the background-job dead-letter list, the exchange-rate refresh, the PostgreSQL management view and
the connected Obsidian vault sync. On a shared install these are available only to the owner of
the **home tenant** (the install's first active tenant, which is yours as the operator). A client's
owner is refused on each of them, so a client admin never operates on the whole install. On a
single-tenant install the tenant's owner keeps all of them. The shared document vault is also
separated per client: a client reads only `tenants/<tenant-id>/...` plus its own
`customer-documents/<tenant-id>/` and `twin-sources/<tenant-id>/` folders, while every other
top-level vault file (including everything written before this release) is treated as shared
content belonging to the home tenant.

**Moving a client's older vault notes (since v4.89.3):** notes written for a client before
v4.89.0 sit at the top level of the vault and therefore look like shared content, so the client
cannot read them. As the home tenant's owner (signed in with a real account) call
`POST /api/obsidian/relocate`. By default it is a dry run that lists what would move. With
`apply=true` it moves a file into `tenants/<tenant-id>/` only when the database shows exactly one
non-home tenant owning it and the destination is free, updates the stored paths in the same step,
and writes an audit entry. Files of the home tenant, files claimed by several clients, records
without a tenant, missing files, symbolic links and anything outside the vault are left in place and
reported. Work is done in batches of up to 500 (`limit`, then `after=<next_after>` until it is
null); running it again is harmless.

**Roles inside each client's tenant (v4.87.0):** a client's users are owner, operator, or
observer. Observers are read-only everywhere. Operators do daily work (tasks, notes, customers,
departments, campaigns, leads) but do not approve actions or execute agents. Owners alone can
hard-delete or cancel higher-impact records (missions, projects, contracts, deals, subscriptions,
budgets, resources, policies, workflows, venture domains, voice profiles) and rotate secrets or
keys; those controls appear disabled with "Owner role required" for everyone else, and a refused
action is rolled back with the reason shown. The full matrix is in `docs/12_AUTH_AND_SESSIONS.md`.

**Since v4.49.0**, `GET`/`PATCH /api/tenants/current` and the GDPR erase/export, agent execution,
prompt version, customer record, and backup-file-listing endpoints all resolve strictly to the
calling session's own tenant.

For the full mechanics (the `erezagent_app` role, the session GUC, per-user isolation within a
tenant), see `ADMIN_GUIDE.md` sections 2 and 9.

---

### Webhooks, project folders and install-wide settings on a shared instance (since v4.91.0)

- **Lead webhooks:** on an instance with more than one tenant, Twilio, Meta Lead Ads and LinkedIn webhooks use
  `/api/webhooks/<provider>/<tenant-id>` so each lead lands in the right client. The bare URL is accepted only on a single-tenant
  instance. A WhatsApp number or Facebook/Instagram page can be connected by one tenant only, and events for an id claimed by more
  than one tenant are ignored.
- **Project folders:** each client's project folders are separate; a project name another client already uses gets a distinct folder
  name, and file, shell, sandbox, scan, download and deployment actions refuse another client's folder.
- **Install-wide settings:** diagnostics, model routing overrides, revisions of shared platform skills, the licensing catalogue,
  Value Hunter scanner settings and the shared action catalog are changed by the instance administrator only; a client's owner
  cannot change them. Routing overrides and shared-skill revisions are sent for approval and applied after the owner approves
  them in Self-Improvement.
- **Live events and chats:** the live event stream checks the signed-in user when it connects and every five minutes, and chat
  conversations are private to the user who started them.

## 4. Per-Client Branding & White-Labeling

### Platform-level branding

For a fully white-labeled deployment (Option A, or a shared instance you're rebranding entirely):

1. Replace `frontend/public/logo.svg` and `frontend/public/favicon.ico`.
2. Update the header text in `frontend/src/components/layout/Sidebar.tsx`.
3. Set `COMPANY_NAME` in `.env` (used in email templates).
4. Rebuild: `docker compose build --no-cache frontend`.

### Per-client portal branding

Each client (venture) gets its own brandable support and booking surface without touching code --
their own logo, color, and display name, so a client-facing page never looks like it's shared with
another client on the same install. Configure this from that venture's **Support** workspace: a
dedicated URL per venture, a custom domain, or an embeddable widget for the client's own site.

**Self-service custom domains (since v4.42.0):** a client's booking page (`/book/{venture_id}`) and
support page can each claim their own domain without any DNS work on your side -- the client points
one A record at your server and adds one TXT record that proves they own the name (the exact record is
shown on the domain's card, since v4.91.0). After the record is verified (Verify now, or the automatic check
about every 30 minutes) the certificate is issued on the first visit to the domain, once `PUBLIC_SERVER_IP` is
set. The first organisation to prove a name owns it, so a claim made by someone who does not control the DNS
cannot block your client. This is the right path whenever a client manages their own DNS. If you
host a client's DNS yourself, the Support workspace's own domain field still works for that case,
but does need a Caddy config entry and restart on your end -- see `ADMIN_GUIDE.md` section 2.

### Per-client AI identity (Persona, since v4.44.0; self-service UI since v4.46.0)

Each client's support and booking portal can present a configured **External Representative**
identity instead of a generic "Support" label -- a name, avatar, tone, and language, scoped to that
one venture (and optionally to a specific customer or project within it). Configure identities from
**Persona Management** (Knowledge & Learning group) -- self-service, per client, and cross-tenant
write-safe: the system independently verifies a representative you create actually belongs to your
own tenant's venture/customer/project before it's ever saved, so you can never accidentally create
an identity that points at another client's data even by supplying its ID directly.

A Persona is identity and presentation only -- assigning one to a client-facing agent never changes
what that agent is allowed to do or which connectors it can reach; every authority decision still
comes from Agent Policies and approval gates, completely independent of branding. Like every other
Persona change, going live requires your (owner) approval. See `ADMIN_GUIDE.md` section 4 for
the full mechanics, including Hebrew/RTL rendering for a client whose configured language needs it.

---

## 5. Per-Client Support Operations

**Customer Operations** gives every client their own real, brandable support portal (see section 4
for the branding side) with case data isolated the same way every other tenant-scoped resource is.

- **Separate URL per venture**, with its own case data.
- **Telegram notification on every new case**, so you never miss a new ticket across your book of
  clients.
- **Email routing** supports either a dedicated mailbox per client or one shared mailbox with
  alias-based routing -- useful when you don't want to provision a mailbox per client.
- **Support Analytics** (Governance group, venture-filterable) -- case volume and SLA performance
  across your whole book of clients, or drilled into one.
- **AI-drafted responses and escalation** for Tier 1/2 tickets, tenant-scoped, always through the
  same approval gate as any other agent-drafted client communication.

### Per-Client CRM Workspace & Document Collection (since v4.78.0)
CRM Workspace and Document Collection are venture-scoped like everything else here -- a checklist,
requirement, or uploaded file created for one client venture is never visible from another's, and
the customer portal upload/status view re-verifies ownership on every request rather than trusting
a session token alone (a real adversarial test proves this closed). A WhatsApp connector can be
connected per client venture (its own access token + phone number) or account-wide if you run one
shared number across clients who don't need their own. Useful for client onboarding paperwork
(engagement letters, KYC/compliance documents) without building a separate intake process per
client.

### Document review and venture staff workspace (v4.86.0-v4.87.0)
A client's venture staff sign in to a focused workspace (My Work, Business Records, Document
Review, Incidents, Appointments) limited to their own venture. In Document Review, staff and
owners can open a submitted file with **View file**; the download is limited to the caller's own
tenant and venture and every download is audit-logged. Business Record file fields have a
matching Download button. Each client venture also has a **Venture Commerce** page for its own
products, subscriptions, and refunds toward its own customers, kept separate from your platform
billing.

### Customer Operating Intelligence across your book of business (v4.79.0)
The portfolio-wide **Needs Attention** view (in CRM Workspace) is filterable by venture, so you can
see every client-venture's own open findings (churn risk, renewal approaching, stalled deal,
overdue document, SLA breach) without them ever mixing across clients -- the same explicit
venture/tenant scoping every other cross-client view in this guide already follows. A governed AI
Customer Brief is available per end customer, and Executive Office can answer questions about a
named end customer even when you don't have their workspace open.

---

## 6. Per-Client Appointment & Engagement (since v4.41.0)

Running scheduling for several client ventures instead of one business, every client gets its own
appointment types, availability windows, and bookable resources, but you manage all of it from the
one place you already manage everything else.

A client's own real public booking page (an extension of the support portal you've already set up
for them, not a second thing to configure) takes external meeting requests directly, checks that
client's connected Google Calendar for conflicts, and hands out a real Google Meet link -- no
juggling a separate scheduling tool per client account. Zoom and Microsoft Teams register per
client and are ready the moment you have credentials for them.

Before a client meeting, an Engagement Brief pulls from that client's own history, so you (or the
teammate covering that account) walk in prepared even if you weren't the one who booked it. After
the meeting, structured outcomes and a drafted follow-up email land on that client's own record and
still go through the same approval you already require before anything is sent.

**Appointment Intelligence (since v4.43.0):** meetings can link to a client's real business context
-- venture, project, customer, lead, or support case -- with ErezAgent proposing the link (never
applying one without your confirmation). Set an objective when a meeting is booked and get an
honest answer afterward on whether it was actually achieved; a report on the Appointments screen
shows which meeting types actually convert to pipeline value, broken down by venture -- useful for
showing a client (or yourself) which engagement types are worth the calendar time.

**Self-service booking domains (since v4.42.0):** a client's booking page gets the same
self-service custom-domain path described in section 4.

---

## 7. Per-Client Security (since v4.40.0)

Managing several client environments means one **Security Passport** per client product (plus your
own platform) -- a single honest PASS / WARNING / FAILED / NOT_TESTED status you can check before a
client conversation instead of digging through raw scan output. It covers threat modeling, code/
dependency/secret/container scanning, software bill of materials, release signing, adversarial
tenant-isolation testing, and open findings. Browse every client asset's passport from **Security
Assurance**'s Portfolio tab.

A real SBOM plus a signed release-provenance record is available per client build, ready to hand a
client's auditor directly.

When an actual incident happens on a client's environment, governed containment actions -- revoke a
credential, disable a user, rotate a secret, disable a route, roll back a deployment -- are
available per-asset, and every one of them still requires your explicit approval before it
executes. No separate incident-response tooling is needed per client.

**Gates on client-delivered code (since v4.20.0):** every project you build for a client gets a
real security scan (Bandit/pip-audit/npm-audit/secret/Dockerfile) automatically at the end of every
build round, tenant-scoped like everything else. Formal acceptance and external publish are both
gated on that scan -- a project with open critical findings can't be accepted or published without
an explicit, permanently-recorded reason from your side. Secure-coding standards are injected
directly into the prompts a client's project is built from.

For the full Security Assurance mechanics (scanners, gate rules, exceptions), see
`ADMIN_GUIDE.md` section 9.

---

## 8. Per-Client Customer Solution Delivery

When a client sends their own requirements document, ErezAgent turns it into a governed
solution-delivery mission scoped to that client's tenant: requirements extraction, feasibility
classification, a technical blueprint with real Executive Board approval, delivery-team assembly,
work-item generation from the approved blueprint, execution, per-requirement test evidence gating
formal acceptance (owner-waivable with an audited reason), and deterministic requirement
invalidation on change requests.

Delivery finishes with either a real approval-gated SSH deployment to the client's own server, or
an export bundle (Docker/Kubernetes) plus ongoing maintained-asset tracking (SLA, support
responsibility, operating cost). Every step -- including every deploy -- still needs your explicit
approval; nothing ships to a client's environment automatically. Kubernetes deployment specifically
remains a manual, export-only process, since no Kubernetes API automation exists yet.

**Android Customer Delivery (since v4.39.0):** when a client asks for a native Android app, it goes
through the exact same governed flow (requirements -> blueprint -> Board approval -> build), so it
shows up on that client's own Customer Asset tab and acceptance record like any other deliverable.
Signing credentials are stored per-project and are owner-only to read or manage -- one client's
keystore is never reachable from another's project, even under the same tenant; every read returns
only a fingerprint, never the raw key.

**Design consistency per client (since v4.15.0-v4.16.0):** each client project keeps its own design
language across missions, gets a real accessibility scan, a responsive check across device sizes,
and a visual-regression comparison automatically. Critical design problems block a mission from
being marked accepted until resolved. If a specific client already uses Figma, you can connect that
one client's own token from their project's settings -- per-project, never shared across clients.

**Client contracts (Legal skills, since v4.33.0):** the Contract Intelligence page covers both a
jurisdiction-general pack (mutual NDA, SaaS MSA, SaaS ToS, vendor/reseller agreement review,
contractor IP-assignment review, trademark clearance, open-source license compliance) and an
Israel-specific pack, available on every client venture. An **AI Review** action runs a chosen
review against a client's actual contract text and shows the result on the page.

**Configuration-first delivery with Build My Business (since v4.88.0):** many client requirements are about how the
business runs (stages, documents to collect, who can see what, portal pages, reminders), not a new application. After
you approve a client's requirements, the mission's **Business Twin** tab sorts each one by what it takes to deliver:
**A** existing capability or configuration, **B** a template covers it, **C** a small extension, **D** new development,
and proposes a model for the client's venture. The client's owner reviews it in Build My Business exactly like a model
built from documents, with an impact analysis, one approval and a rollback. If every approved requirement is A or B, no
standalone application is generated; only the C and D requirements go to the blueprint and build path above. A client who
has no formal requirements document can upload their own paperwork, or describe their business, in the same screen.

**Solution Templates for consultants (since v4.88.0):** once a client's model works, the owner can promote parts of it to
a template. Promotion removes names, contact details, documents, credentials, customers and staff automatically, shows a
report of what remains, and requires the owner to acknowledge it before publishing. Installing a template into a venture
creates a draft that still goes through preview and approval, so nothing reaches a client's customers unreviewed. Templates
belong to the tenant that created them and are never visible to another tenant, so a template you build in one client's
tenant cannot be installed into another client's. If you run many clients as ventures inside one tenant (your own
consultancy tenant), a template can be installed into any of those ventures. Templates provided with the platform are
readable by every tenant and cannot be modified by one.

---

## 9. Fleet Management

### Upgrading multiple client deployments

Installation and upgrade mechanics -- prerequisites, the Web Installer Wizard, manual setup, and
every documented upgrade path -- live in `docs/DEPLOYMENT_GUIDE.md`. This guide doesn't restate
those steps. One discipline specific to running a fleet: **refresh your local installer folder
before upgrading each client**, not just the client's running containers -- the upgrade scripts
themselves change between releases, so an installer folder cached from an earlier client's upgrade
can skip steps added since (for example, the automatic repair of data-volume ownership and the
appending of newly introduced `.env` settings). `git pull` in your `ErezAgent-Deploy` clone before running
`upgrade.sh`/`upgrade.ps1` against the next client on your list.

**Upgrading a client to v4.89.0:** nothing manual. Migrations 529-545 run on start, and a blank `LICENSE_PUBLIC_KEY` is filled with the vendor key. Tell each client's owner that billing changes are owner only (operators read, observers have no billing access), that a license pasted in Settings -> Platform License takes effect immediately, and that the API may take about two minutes to become healthy after the upgrade. A rollback now also reverts the database schema (after confirmation and a backup), so keep the pre-upgrade checkpoint. If you rotate your license signing key, do it only after every client has upgraded to v4.89.0 or later, because older installs verify only licenses signed directly by the original key.

**Upgrading a client to v4.88.0:** nothing manual. Migrations 508-528 run on start and four optional `TWIN_*` settings
are appended to the client's `.env` with defaults. After the upgrade, tell each client's owner that AI analysis of documents
is off until they grant consent (features they start themselves on their own documents keep working, with identifiers always masked first), that the core agents now keep per-tenant settings, and that a workflow marked as
requiring approval now waits for that approval (only an owner can change that setting). For a client that must keep all processing on its own hardware, set up
Ollama for them and run `ollama pull qwen2.5vl` for scanned pages and images (see `ADMIN_GUIDE.md` section 23).

Each client's daily data-retention job (04:00 UTC) only deletes aged data after a backup completed
within the last 48 hours (`RETENTION_REQUIRE_BACKUP_WITHIN_HOURS`); if you back clients up outside
ErezAgent, mark backups as managed externally in that client's Backup Center. See
`docs/DEPLOYMENT_GUIDE.md`.

Before running an upgrade against a client, check **Update Planner** (Platform Management) for that
deployment -- it tells you the downtime estimate, backup readiness, migration requirement, and risk
level before you commit to the window.

### AI capability governance per client

New AI tools and integrations (MCP servers, external APIs, CLI utilities) don't appear live in a
client's system on their own -- each one moves through a governed lifecycle (discovery ->
evaluation -> recommendation -> **approval** -> acquisition -> validation -> continuous review)
and only becomes active once you've explicitly approved it. The system can also propose a new
capability on its own when a client's department consistently misses a KPI target, with a 90-day
check afterward on whether the acquisition actually moved the metric. Review and approve from **AI
Resources** (Platform group) on that client's instance.

---

## 10. Billing & License Management per Client

ErezAgent has two independent licensing layers -- keep them mentally separate; both are covered in
full in `ADMIN_GUIDE.md` section 11.

- **Your own platform license** governs what your own ErezAgent instance (or each client instance,
  under Option A) can do.
- **Licensing Department** (Revenue & CRM group) is the platform-owner tool for distributing
  ErezAgent to your own customers: issue and track customer licenses, monitor activations, and
  review license events. **Platform Licenses** (its owner-only tab) is where you issue unlimited or
  special licenses to trusted partners -- deliberately restricted to the owner role given the
  financial and legal weight of that action.

### Provider Console -- managing downstream managed installs (since v4.51.0)

If you're running an Option A fleet (a fully separate self-hosted install per
client -- own DB, own containers, no shared database and no RLS tenant
multiplexing between you and a downstream install, exactly as described in
section 2), **Provider Console** (a tab on the Licensing Department page,
alongside Platform Licenses) gives you one place to register, license, and
meter those downstream installs -- it does not change that deployment model;
it just adds real tracking and licensing on top of it.

- **Register a downstream install** with its fingerprint, company name, and
  admin email. Registration issues that install a real, Ed25519-signed
  license -- signed with **your own tenant's Layer 2 signing key**
  (`lic_signing_keys`), not the platform owner's Layer 0/1 key. These are
  deliberately separate trust domains: your signature proves "this is a
  legitimate licensee of *my* product," never "this is a legitimate ErezAgent
  install" (that's Erez's own Layer 0/1 signature, covered in
  `39_PLATFORM_LICENSES.md`). Generate/rotate your own signing key from the
  Signing Keys panel on the same tab before registering your first install --
  an install registered before a key exists gets an unsigned license.
- **Record usage reports** (customers, ventures, active users; owner only) and
  **deactivate** a managed install (status flip plus license revocation, audited --
  history is never deleted) from the same tab's install list. Each install shows
  its usage report history and when it last reported. Usage reports are recorded by you or
  imported from a file the customer sends. An installation that has been connected to your
  instance through the Provider Fleet page (below) reports its operational status automatically instead; that
  report carries no usage figures used for billing.
- As of v4.77.0, registering your first downstream install marks **your own**
  subscription (the one billed on your `is_provider_plan=true` plan version)
  `commerce_mode='msp'` for real reporting visibility -- this is a labeling
  change only, never a payout mechanism: a downstream install is a physically
  separate self-hosted deployment with no per-transaction commission
  relationship to compute, so nothing about how it bills you changes.
- **Purchase Webhook panel** -- configure the secret your own marketing site
  uses to trigger real license issuance for your products via
  `POST /api/licensing-dept/purchase-webhook`, HMAC-signed. This is the same
  per-tenant Layer 2 purchase-webhook mechanism covered in
  `37_LICENSING_DEPARTMENT.md`, surfaced here alongside Provider Console since
  most MSPs configuring downstream installs also wire this up at the same
  time.

**Volume-tier billing.** If you've negotiated per-tier pricing for your
downstream fleet, it's stored on your own `commercial_contract_overrides.
volume_tiers` (an Enterprise/Provider negotiated term, not part of the shared
plan catalog -- see `DEVELOPER_GUIDE.md`'s Commercial Licensing & Billing
section for the catalog-vs-contract-override distinction). The Provider Console's
**Volume tier charge** panel shows the tier that applies to your active managed
installs and the resulting amount; it is a preview, not an invoice. The same
figure is available with:

```bash
curl -H "Authorization: Bearer $EREZAGENT_API_KEY" \
  http://localhost:8000/api/licensing-dept/provider/volume-tier-charge
```

### What your customer sees (v4.94.0)

A customer owner connects an enrolled installation in **Settings > Provider / Managed Service**: they paste the
enrolment string you give them, check your address, tick the consent box beside the lists of what is and is not shared,
and connect. They can see the exact last report, pause, resume or disconnect at any time, and answer your notices
(I will update myself, Remind me later, Request assistance, Request Professional Services). If a later release widens the
list of shared items, reporting pauses until they confirm again. Tell customers this before they connect; nothing is sent
before they do.

### Provider Fleet page -- status of the installations you enrolled (since v4.94.0)

On a platform-owner instance, **Fleet** shows the operational state of the installations you enrolled:
version, component health, security summary, AI and voice provider status, readiness, and the licence and
subscription state each installation reports. The commercial block beside it comes from your own records
and is labelled as such. Nothing is installed or changed remotely; the customer keeps full control of the
server and performs every update.

- **Enrol an installation.** Create it for a platform customer; Fleet shows a one-time enrolment string
  that expires after 7 days and works once. The customer's owner pastes it in Settings -> Provider /
  Managed Service and ticks the consent box next to the list of what is shared. Nothing is sent before that
  click, and the customer can pause or disconnect at any time.
- **Watch and filter.** Nine filters show what needs attention: offline, unhealthy, update available,
  unsupported version, security attention, AI provider issue, licence expiring, subscription attention and
  support required. An installation is offline after three missed reports (at least 30 minutes).
- **Notices.** Send a plain-text notice (update, security update, information or service offer) to one or
  many installations. The customer answers with "I will update", "remind me later", "request assistance" or
  "request professional services"; the last two open a support case for you.
- **Packages and cases.** Service packages are labels you create (the page offers the standard four:
  License Only, Standard Support, Managed ErezAgent, Professional Services / Custom). A package never
  grants remote access. Cases use the normal support case tools.
- **Service report.** A report for a period states only stored facts, marked "reported by the
  installation", in English, Hebrew or Arabic. It makes no uptime or service-level claim. Sending it by
  e-mail needs your approval.
- **Open a case, add recipients.** From an installation you can open a case (with a professional
  services flag) and see who it waits on; from a notice you can add more installations to it.

#### Provider staff roles -- who can do what (since v4.94.0)

You can give your team their own sign-in to the Fleet page without giving them your owner account.
Invite each person from **Fleet -> Provider staff**, choose a role and the customers they may see. A
person sees only the customers you assign; with none assigned they see nothing. You can change a role or
the customers at any time, and deactivate a person, which ends their sessions at once.

| Role | Can see | Can do |
|---|---|---|
| NOC / Operations | the fleet and its report history | send update and information notices |
| Support engineer | the fleet, contacts, cases, reports | open cases, send update and information notices, generate reports |
| Security operator | the fleet and its report history | send security notices |
| Billing operator | the fleet, contacts, commercial records, packages, reports | assign packages, propose a report e-mail (you approve it) |
| Auditor | everything above | nothing; read-only |

Only you can enrol or revoke an installation, edit installation details, manage packages, link an
existing case, send a service offer or manage staff.

### Billing your own customers and clients (since v4.89.0)

- **Billing method per venture.** Each venture (or the whole workspace) bills by **metering**
  (no payment gateway; you record payments yourself) or through a gateway you connect in
  Platform Management -> Billing Setup (Stripe, PayPal or Rapyd for plans; Pelecard only for
  Revenue Center deal payments). Changing the method is an owner approval. In Venture Commerce a
  metering venture creates subscriptions without a checkout, closes each billing period and
  creates the recurring draft; you renew manually. Rapyd and Pelecard are not available there.
- **Usage statements and invoice drafts.** From the usage ledger, generate a period statement
  per client or venture, finalize it, draft the invoice, record the payment (or import a bank
  CSV), and send the PDF or DOCX by governed email. Prices come only from the unit prices you
  set. These are usage statements and invoice drafts, not tax invoices: issue the legally valid
  invoice from your accounting system. Signed webhooks, scoped tokens and exports under Revenue
  Center -> Billing -> Integrations feed that system (`ADMIN_GUIDE.md` section 11).
- **Customer records for downstream licensing.** On your platform-owner installation, Platform
  Licenses -> Customers holds contracts, invoice drafts, payments, renewals and license updates
  for the customers you license directly. Self-hosted installs report usage as a file: the
  customer exports it from Revenue Center -> Billing -> Usage (aggregate figures only, no names,
  attested by the customer, never sent automatically) and you import it. A figure that differs
  from an earlier import for the same month is held for review.
- **Chargeback** below remains the way to attribute AI spend to a client's departments.

### Department / cost-center chargeback

Attribute AI spend to a client's individual departments, and generate chargeback reports for
billing: create a department with a name, cost-center code, and monthly budget from **Custom
Departments** (Governance group), assign agents to it, and pull a chargeback view -- gross spend,
savings attributed, net cost, and each department's share of total spend -- from **AI Spend
Intelligence** (Cost & Finance group), filterable by period. The net-cost figure (gross minus AI
savings) is the billable amount per department for a cost-plus arrangement.

### QBR and renewal reporting

From **MSP Operations** (see section 11), QBR briefings, SLA reports, and renewal outreach are
written by a real AI pass over each client's actual account data, not a fixed template -- including
a **Draft Renewal Outreach** action that produces a ready-to-send message for an at-risk client
renewal.

---

## 11. Monitoring Your Fleet

### MSP Operations dashboard

**MSP Operations** (`/msp`, Cloud Ops group) is your cross-client portfolio view (operators can record client health entries; observers are read-only):

- Client portfolio grid with health score, SLA compliance % (as entered), open/critical ticket counts, ARR, and
  renewal risk per client.
- Active tickets, escalations, and SLAs.
- Agent activity log per client context.
- QBR briefings, SLA reports, and AI-drafted renewal outreach (section 10).

### Support Analytics

**Support Analytics** (Governance group) is venture-filterable case volume and SLA performance --
your whole book of clients, or one client drilled into.

### Infrastructure monitoring

**VMware VCD** (`/vmware`) and **OpenStack** (`/openstack`), both in the Cloud Ops group, give you
VM lifecycle management (start/stop/snapshot/delete, all approval-gated) and per-project health and
quota monitoring, for clients whose infrastructure you manage directly.

**FinOps** (`/finops`, Cost & Finance group) covers cloud spend by client, by service, and by
region, with anomaly detection and budget enforcement gates.

**Security Assurance** (Research & Signals group) surfaces CVE feeds, scanner results, and policy
violations per client environment -- see section 7 for the per-client Security Passport.

### Cross-fleet observability

Prometheus (`/metrics`), Grafana dashboards, and structured JSON log shipping work the same way for
a shared multi-client instance as for a single-tenant one -- see `ADMIN_GUIDE.md` section 17
for the metric names, dashboards, and alert rules. Point your existing SIEM or Grafana at the same
endpoints regardless of how many client tenants the instance serves.

### Executive Board per client

Use the **Objective Center** to track strategic objectives per client engagement -- create an
`operational_automation` or `internal_business_tool` objective for the engagement, run a **Board
Deliberation** before committing to a new client initiative, and use the **Board Room** tab's
deliberation history as source material for QBR presentations. Each client's Executive Board reads
only their own tenant's **Company Foundation** (Constitution + Active Mission) -- see
`ADMIN_GUIDE.md` section 3 for how to set this up per client, whether you're filling it in on
their behalf during install (Option A) or they're setting it themselves after logging into their
own tenant (Option B).

---

*(c) 2026 Erez Rozenbaum. All Rights Reserved.*