Mailbox

Overview

The Mailbox plugin (/plugins/mailbox/) is the platform's self-hosted email subsystem — the receiving counterpart to outbound sending (SystemMailer). Its first feature is forwarding: admins create aliases (e.g., [email protected]) that forward incoming email to real addresses.

Postfix receives inbound mail, pipes it to a PHP handler, which looks up the alias and relays it through the selected outbound provider (see Forwarding relay).

Self-hosting here means inbound. The plugin owns receiving — MX, Postfix, the pipe handler, verification milters. Everything it sends (forwards, replies, composed mail) leaves through the platform's configured outbound provider, the assumed path for all outbound mail (see the outbound doctrine in Email System). Delivering directly from this box's own port 25 to recipient mail servers is an advanced setup a deployment must deliberately pursue (cloud egress unblock, PTR, IP reputation) — never a required step of mailbox setup.

Features: multiple domains, multiple destinations per alias, catch-all addresses, SRS for SPF compatibility, inbound authentication results (SPF/DKIM/DMARC) read from the verifying MTA / provider, outbound DKIM signing (opendkim), per-alias and per-domain rate limiting, RBL spam filtering, inbound email logs with admin viewer, live DNS validation.

> Putting a domain on hosted mail: the sender-identity layout, the record > set, the cutover order, and the provider and DNS behaviours that produce a > setup which looks correct and is not — see > Bringing a Domain onto Hosted Mail.

Installation

Prerequisites

Postfix (with the postfix-pgsql map driver), opendkim and opendmarc are installed and configured by provisioning/install_email.sh — run it once per deployment (see Server Setup below). It assumes one Joinery site per host; that host may be a Docker container or bare metal.

> Setup status on the Plugins page. Once activated, this plugin declares three provisioners, so the admin Plugins page (/admin/admin_plugins) reports whether its runtime dependencies are working: a missing inbound mail server shows Needs setup with the provisioning/install_email.sh fix command; a down or misconfigured outbound relay shows Needs setup with the reason; and missing MX/SPF DNS records on any enabled inbound domain show Needs setup listing the affected domains. See the "Declaring Host Provisioners" section of docs/plugin_developer_guide.md.

Enabling

1. Activate the plugin in Admin > System > Plugins
2. Run update_database from admin utilities to create tables and run migrations
3. Mailbox appears under Emails in the admin sidebar — it opens on the Setup tab

The receive-mode choice (relay or direct)

One deployment-wide fact shapes every domain's DNS prescription: does mail come straight to this server, or does a relay front it so the server's address stays hidden?

It is a setting, not a gate. An undecided deployment receives directly and works; the choice lives in the Setup tab's Advanced section and can be changed at any time. A relay is only load-bearing at the Fortress security level, so the answer is asked for where it becomes true — raising a domain to Fortress — rather than in front of every mailbox page before any domain has a level.

The control is a brief pros/cons comparison (setup effort, whether the server's address is public or hidden, and that a relay is *required for the Fortress email security level) with one choose button per column. The choice belongs to the admin: a relay provisioned as part of setup does not decide it.

mailbox_receive_mode() (includes/receive_mode.php) resolves the mode:

1. The stored choice (mailbox_receive_mode setting) → its value. Choosing relay redirects to the Setup tab's Relay section; choosing direct redirects to Accounts to add the first domain (with a pointer to remove any provisioned relay).
2. Live domains with no stored choice → the mode reports what the deployment is actually doing (live relay row → relay, else direct).
3. Otherwise '' — undecided, which every consumer treats as direct.

• Receiving (always): the mailbox's domain DNS verified for correctness, not just presence (MX target actually resolves to this server, SPF authorizes the IP, DMARC published), that the domain is registered, that inbound mail is being authentication-verified (opendkim-verify + opendmarc), that the alias resolves, and an end-to-end proof — send a real message and watch it land in the logs. For an IMAP-source mailbox there is no MX/host stack, so this group instead reports the feed's connection state and last fetch.
• Forwarding (only when the mailbox forwards): the outbound relay, SRS, and DKIM signing — the checks that matter when mail is forwarded back out.

• On/off is derived, never a toggle: the machine sender is on when the defaultemail setting's domain is a proper subdomain of the focused domain. The setting that controls where system mail sends from IS the switch, so the card can never disagree with what happens at send time.
• Off, and eligible: a grey OPTIONAL card on the domain system mail sends as, offering the setup. Other domains show nothing.
• Off, and blocked: when EmailSender::transactionalSendBlocker() reports the system sender can never send (a protected identity, typically), the card is a REQUIRED FAIL — mail is being dropped and nobody chose that. An empty or invalid defaultemail matches no domain view, so that blocker surfaces as a plugin.system_mail row under Advanced instead.
• On: the sub-checks are REQUIRED — provider registration in a usable state (registered-but-unverified fails), the provider's DKIM records live in DNS, SPF on the machine domain with the provider mechanism and a strict -all terminal (extra mechanisms pass; a softer terminal fails), the machine domain not itself protected, and an informational Reply-To row (defaultreplyto unset is a hint, never red). A Send a test email action proves the whole ambient path with one real send to the operator; with email_dry_run on it says so instead of reporting a suppressed send as proof.
• v1 verifies providers that implement DkimRecordSource (Mailgun, SES); other transports render guidance rather than fabricated verdicts.

The send-protection ceremony's readiness rows include System mail has somewhere to go: a warning when defaultemail still sends as the domain being protected, since protection will refuse every automated send. It warns rather than blocks — proceeding past it is an explicit act.

Topology-aware prescriptions

Every prescription derives from the deployment's receive topology, resolved from the MailboxRelay row: colocated (no relay row — the box is the MX), self-hosted relay (mrl_is_hosted = false), or hosted fleet slot (mrl_is_hosted = true). A relay row's existence — enabled or not — flips every prescription to relay targets: the checklist walks the user to the relay end state, so mid-cutover guidance already names the relay. Topology is deployment-level; the security level is per-domain.

Under a fronted topology:

• MX must string-equal the relay's MX hostname (mrl_mx_hostname) and resolve to the relay's public IP. The box's address is never prescribed.
• SPF prescribes the outbound provider's mechanism alone (v=spf1 <mechanism> -all, from EmailServiceProvider::getSpfMechanism()) — a record naming the box FAILS, because it publishes the address the relay hides. Local-sendmail outbound gets no record prescription: that row prescribes switching to an API provider.
• Relay identity rows (the MX hostname's A record, the relay IP's PTR) replace the box's own A/PTR rows. On a fleet slot they are operator-published and render as neutral INFO when missing; on a self-hosted relay the tenant owns the zone, so they are REQUIRED with the fix.
• Domain ownership (fleet only, domain.ownership): the fleet accepts no mail for a domain until a TXT proof is published. The row behaves like every other DNS row — the challenge is filed automatically (at enrollment for every registered domain, and at domain registration while a slot exists), re-verified on every check pass, and shown with the copy-ready TXT record until it goes green. There are no buttons and no claim/verify vocabulary.
• Cutover progress (plugin.relay_enable): relays are born enabled, so this row reports how far the DNS move has come — INFO with the first incomplete reason while MX records move, PASS once every hosted domain's MX targets the relay (and every ownership proof is published). The one bad state — cutover complete while the relay sits emergency-disabled (mail arriving with no consumer) — is a REQUIRED FAIL. Every evaluation records its verdict in the mailbox_relay_cutover_complete setting, which is what the outbound doctrine enforcement and the origin-hidden health check read — a fronted deployment keeps sending the legacy way until the cutover verdict flips, so nothing breaks mid-move and nothing leaks after it.

Settings and diagnostics that are server-wide rather than per-mailbox live behind the Advanced server setup disclosure: the inbound provider picker (mailbox_provider), this server's mail identity — the FQDN (mailbox_mail_hostname, used as the MX target, HELO name, and PTR name) and public IP — the provider's DNS records to publish, and the full inbound health run (every layer: Postfix/pipe transport/domain map/opendkim/port 25, mail identity, domain DNS, plugin config, and end-to-end). Set the mail hostname here once; everything else is autodetected.

The Accounts tree

Emails > Mailbox > Accounts is the single place to see and manage routing: every domain, the mailboxes (aliases) nested under it, how each mailbox routes (stored / forwarded / both), and any IMAP feed pulling mail into it. A domain is either MX-hosted (mail pushed in) or an IMAP source (mail pulled in per mailbox — e.g. gmail.com, no MX needed); both nest identically. The tree is the overview and entry point; + Domain and every Edit open the per-object editor with context pre-filled. Under an IMAP-source domain the mailbox is its feed: Edit opens one combined editor that manages the mailbox name, its reader, its protection level and the IMAP feed together — there is no separate feed object. Hosted (MX) domains keep a distinct + IMAP feed per mailbox, which opens the connect wizard — the feed is created there, attached to the hosted mailbox.

+ Connect a mailbox is how a pulled-in mailbox is created, and the only way (admin_mailbox_connect); + Mailbox on an IMAP-source domain leads there too. See Connecting a mailbox below.

Adding a Domain

The Setup tab can register a domain for you (a one-click action on the "Domain registered" check). To manage domains directly: on the Accounts tab click + Add Domain, enter the name and save — Postfix picks it up immediately (the inbound domain list is read live from the database; no host command, no per-domain Postfix config). Then use the Setup tab to verify and publish the domain's DNS records. + Add Domain creates hosted domains only: an IMAP-source domain (gmail.com) comes into existence through the connect wizard, together with its first mailbox.

Adding an Alias (mailbox)

1. On the Accounts tab, click + Mailbox on the hosted domain you want
2. Enter the alias name, delivery mode, and destinations (for forwarding)
3. Save

Consent creates the mailbox. The flow payload carries the operator's intent — provider, reader, protection level — and no ids, because none of those rows exist yet. InboundImapOAuthConsumer creates the mailbox on success through ImapFeedProvisioner::provision(), which is the one path a pulled-in domain, alias, grant and feed come into being by; the combined editor edits what it made. A grant that carries account_id instead is a reconnect, and stores the token on the feed that already exists.

The address comes from the provider. Google and Microsoft report which account consented (OAuth2Client::fetchIdentity()), and that answer is authoritative — it is the address the IMAP session will authenticate as, so a typed one that disagrees is simply wrong. Where a provider cannot say, the wizard asks, with the grant already held in the session; losing the convenience never loses the connection.

Someone else can sign in. Choosing that creates the mailbox with no token: it appears on the Accounts tree switched off with its normal Connect button, finished later by a permission-10 admin on the owner's device.

Protection is asked before the import, deliberately. Sealing happens per message at store time, so a mailbox set to Private before its archive is imported seals every message as it lands, at no extra cost. Set afterwards, the same end state means the backlog pass rewriting every stored row, 200 at a time, from the browser.

The feed is created disabled and starts fetching when the configure step is finished, so an abandoned flow leaves a mailbox that is visibly not enabled rather than one quietly collecting mail nobody finished asking for.

A connected account is not a hosted domain. Connecting [email protected] creates a domain row for gmail.com marked as an IMAP source — bookkeeping for the mailbox, never an identity this deployment owns (InboundEmailDomain::is_authoritative() answers no). Nothing treats it as hosted here: anyone can still register or set a recovery address @gmail.com, no DNS is checked or prescribed for it, no Joinery Direct signing identity is minted for it, and the messenger does not treat its addresses as local. Mail from the mailbox leaves only through the connected account's own SMTP: when the feed is disabled, unauthorized, or a generic IMAP host with no SMTP, the reader's compose shows the reason before anything is written, the Setup tab's Sending row says the same, and a send refuses with the same words — it never falls through to the site's provider or the relay (MailboxSender::sendCapabilityFor() is the one answer all three read). The setup wizard's Email step counts a syncing connected account as a complete receiving arrangement ("connected account", green), and "connection paused" when the feed is off.

Plus-tagged addresses cannot be connected. A mailbox local part admits no +, so [email protected] is refused with the base address named; connect [email protected] — the provider delivers the tagged form to the same inbox, so nothing is lost.

The configure step asks about sync too. The discovery connection that lists the server's folders also detects its sync capabilities, so when the server can keep two copies in step the wizard offers the same Keep in step with the original choice (Off / Read-only / Two-way, with the deletion and compose toggles) that the mailbox editor carries — see Sync. When the server cannot, the step says plainly that the feed is a one-way import that never changes the original.

Server Setup

On apt-based systems, run provisioning/install_email.sh as root, once per deployment. It installs Postfix, postfix-pgsql, opendkim and opendmarc and applies the fixed base configuration, idempotently:

• the joinery pipe transport in master.cf;
• virtual_transport = joinery, inet_interfaces = all, a safe mydestination, and RBL smtpd_recipient_restrictions;
• virtual_mailbox_domains wired to a PostgreSQL map (see below) so Postfix reads the live inbound-domain list straight from the database;
• opendkim config — inet socket on localhost:8891, Mode sv (sign and verify), empty key/signing tables, and an AuthservID matching the configured mail hostname;
• opendmarc config — inet socket on localhost:8893, SPFSelfValidate true, RejectFailures false (stamp-only, never reject);
• both Postfix milters, in order: `smtpd_milters = inet:localhost:8891, inet:localhost:8893` (opendkim first so opendmarc can consume its DKIM result), with milter_default_action = accept so a down/keyless milter never blocks mail. Received mail is thereby stamped with an Authentication-Results header the app reads for SPF/DKIM/DMARC (see Inbound authentication below).

The app never computes these verdicts itself. SPF and DMARC are structurally impossible to compute at the PHP layer: SPF is a function of the connecting client IP evaluated against the sender domain's record, and the inbound Postfix pipe (utils/inbound_email_handler.php) only ever receives the raw MIME on stdin plus the envelope recipient — the connecting IP is known only to smtpd, before the pipe. So the verdicts come from whoever ran the inbound MTA, in one of two trusted forms, resolved by InboundEmailRouter::readAuthResults() in precedence order:

1. Webhook provider verdicts. A provider that received and verified the message upstream (Mailgun, SendGrid, SES) returns its SPF/DKIM/DMARC results from handleInbound() as an auth array. The webhook dispatcher threads that into processEmail(), and the router records it with iem_auth_source = the provider key (mailgun / sendgrid / ses).
2. Authentication-Results header. For the self-hosted Postfix path, the verifying milters — opendkim in verify mode and opendmarc (SPFSelfValidate) — evaluate SPF/DKIM/DMARC on receipt and stamp an Authentication-Results header with our AuthservID. AuthenticationResults (in includes/) parses that header and the router records iem_auth_source = 'milter'.
3. Relay stamps. Under a fronted topology the relay is the verifying MTA: its own milters evaluate the message on receipt, and the sealer carries every Authentication-Results line into the .meta sidecar. InboundEmailRouter::authFromRelayMeta() reads them when the message is pulled and records iem_auth_source = 'relay'.

• WARN — the selected provider has no inbound verification path at all, or the provider is Postfix but verification is broken (milter unreachable, opendmarc missing, config drift). Fix: run install_email.sh, then send a test message to confirm an Authentication-Results header appears.
• WARN (fronted) — mail is arriving from the relay but none of it carries a verdict. The relay is delivering while its stamps are being refused, which is what an authserv-id that is not the relay's mail hostname looks like, or a relay whose own milters are stopped. Fix: re-run the relay provisioner on the relay host, then send a test message. This case is called out separately because the alternative — reporting it as nothing has arrived yet — hides a live defect behind a to-do.
• INFO (neutral) — the verifier is in place, but no verdict-carrying mail has arrived yet to confirm it. For Postfix this also covers a host whose config isn't readable by the web user; for a webhook provider (Mailgun/SendGrid/SES) or a relay it simply means no message stamped with that iem_auth_source has been received yet. We legitimately can't tell yet — not an alarm.
• PASS — recent mail carries verdicts (iem_auth_source = milter for Postfix, relay under a fronted topology, or mailgun / sendgrid / ses for a webhook provider). The behavioral signal (verdict-carrying mail actually seen) is authoritative, because a milter can be wired-but-unreachable; for Postfix the config probe only enriches the reason.

virtual_mailbox_domains = pgsql:/etc/postfix/joinery-domains.cf

@                 MX   10  mail.yourserver.com.
@                 TXT  "v=spf1 ip4:YOUR_SERVER_IP -all"
mail._domainkey   TXT  "v=DKIM1; k=rsa; p=YOUR_PUBLIC_KEY"

virtual_transport = joinery
virtual_mailbox_domains = pgsql:/etc/postfix/joinery-domains.cf
inet_interfaces = all
mydestination = localhost, localhost.localdomain

# opendkim (verify) then opendmarc — order matters; accept on milter failure.
milter_default_action = accept
smtpd_milters = inet:localhost:8891, inet:localhost:8893
non_smtpd_milters = inet:localhost:8891

smtpd_recipient_restrictions =
    permit_mynetworks, reject_unauth_destination,
    reject_rbl_client zen.spamhaus.org,
    reject_rhsbl_helo dbl.spamhaus.org,
    reject_rhsbl_sender dbl.spamhaus.org, permit

hosts    = localhost
user     = iemap_<dbname>
password = <the role's password — lives only in this file>
dbname   = <db name>
query    = SELECT ied_domain FROM ied_inbound_email_domains
           WHERE lower(ied_domain) = '%s'
             AND ied_is_enabled = true
             AND ied_delete_time IS NULL

joinery   unix  -  n  n  -  5  pipe
  flags=DRhu user=www-data
  argv=/usr/bin/php /var/www/html/SITENAME/public_html/plugins/mailbox/utils/inbound_email_handler.php ${recipient}

install_email.sh installs opendkim's static config (the inet socket, Mode sv, AuthservID, empty key.table / signing.table / trusted.hosts, and the Postfix milter). Mode sv means it signs outbound and verifies inbound (stamping the DKIM result into Authentication-Results — see Inbound authentication above, where opendmarc adds SPF/DMARC). opendkim runs from first install — keyless for signing until a per-domain key is added, but verifying inbound DKIM immediately — and milter_default_action = accept guarantees a keyless or down opendkim never blocks or defers mail.

> The opendkim.conf the installer writes is keyed on a managed marker. Re-running > install_email.sh re-asserts the managed config — the inet:8891 socket, > Mode sv, and the AuthservID — and realigns Postfix's milter wiring to match, > so a host whose opendkim config has drifted is brought back into line on the > next run.

Generating a key is a per-domain step (a key file on disk plus a DNS record cannot be a database lookup). provisioning/provision_dkim.sh does the whole host side in one idempotent command:

sudo bash plugins/mailbox/provisioning/provision_dkim.sh example.com

It runs opendkim-genkey, appends the key.table / signing.table lines (only if absent), restarts opendkim, and prints the DNS TXT record to publish at mail._domainkey.example.com. Re-running for a domain that already has a key is a no-op that just reprints the record. The Setup tab's "DKIM signing key" check offers this exact command as its fix, and the following "DKIM record published" check then hands you the TXT record as a copy-paste DNS fix.

Forwarding works without a DKIM key; only outbound DKIM signing is affected.

The Setup tab's DKIM rows follow the signing path (specs/mailbox_provider_dkim.md): the local opendkim key above is prescribed only when the domain's mail actually leaves through local Postfix (colocated deployments). When composed mail rides an API provider — always the case on a relay-fronted deployment with provider outbound, and additionally on colocated deployments whose active provider is API-class — the correct DKIM record is the one the provider issues for the domain, and the row verifies exactly that: providers implementing DkimRecordSource (Mailgun, SES — see email_system.md) report their required records from their own API, and the Setup tab renders one row per record, each checked against live DNS with a copy-paste fix. A domain not registered at the provider gets a row saying so (mail from it fails DMARC alignment until it is added at the provider dashboard); a provider without the capability gets generic guidance naming it. When sent mail leaves through the relay, the row states plainly that sends carry no DKIM signature.

Firewall

install_email.sh runs ufw allow 25/tcp when ufw is active. Bare metal or a container, the site's Postfix owns port 25 on its host.

Container persistence

On a systemd host, install_email.sh runs systemctl enable, so Postfix and opendkim restart on boot automatically — nothing else is needed.

A Docker container has no systemd; its CMD is the init. The Joinery site image handles the mail stack the same way it handles PostgreSQL and cron — by (re)starting it on every container start. The plugin declares install_email.sh as its host_installer in plugin.json, and the CMD runs _plugin_installers_start.sh, which executes every active plugin's declared host installer — for Mailbox that re-applies the Postfix / opendkim configuration and starts both daemons (via the idempotent install_email.sh). The mail packages themselves are baked into the base image. So in a container the mail stack survives a docker stop/start and an image rebuild with no manual step.

This applies to images built from base image version 1.1 or later. An older container keeps relying on a manual install_email.sh run until it is rebuilt and redeployed — base-image changes do not travel through the code-upgrade pipeline. See the mail_stack_container_persistence spec.

Advanced: multi-site host relay (manual, not installed)

A more complex topology — several sites behind one IP, with a host front-relay demultiplexing inbound mail to per-container Postfix instances by domain — is possible but is manual, operator-level configuration. install_email.sh assumes one site per host and does not set this up. If you run it, the host relay (relay_domains, transport_maps) and per-container port mapping are yours to maintain; RBL checks would happen on the host relay only.

Settings

Delivery-policy settings — spam filtering, forwarding limits, the forwarded-From display, and retention/storage caps — are edited on the Settings tab. Server identity and provisioning (provider, mail hostname/IP, SRS, the relay) live on the Setup tab.

/plugins/mailbox/
├── plugin.json
├── data/          — Domain, Alias, Log models (auto-create tables)
├── includes/      — InboundEmailRouter (processing), InboundEmailHealth,
│                    InboundEmailSetupCheck (guided-setup verification engine),
│                    MailboxSpamPolicy (derived spam posture), SRSRewriter
├── utils/         — Postfix pipe script (inbound_email_handler.php),
│                    spam_policy.php (spam posture readout for shell sessions),
│                    managed_domain_prepare.php (make a domain mail-ready and
│                    print its DNS plan, for a management node over SSH)
├── provisioning/  — Host setup: install_email.sh, provision_spam_scanner.sh,
│                    render_pgsql_map.php
├── admin/         — Admin pages (setup, aliases, alias edit, domains, logs)
├── logic/         — Logic files for admin pages
├── tasks/         — Scheduled tasks (relay reconcile, IMAP poll, imports, filters)
└── migrations/    — Settings and menu entry

The Sending route check on the Setup tab verifies the resolved outbound path: when provider relay is active it confirms the provider's own credential is configured (so a healthy API key reads PASS even with empty smtp_*); on the SMTP fallback path it connects to the SMTP host and closes. It is named for what it is — the route outgoing mail takes — and has nothing to do with the ingest relay described under The receive-mode choice; the two are unrelated, and sharing the word relay between them left no way to tell which a row was about.

Knowing a mailbox is unfinished

The Setup tab tells you whether a mailbox is configured correctly, but only if you go and ask it. The Accounts listing badges the ones worth asking about, so a half-finished mailbox does not sit broken until somebody happens to open the page that would have said so.

The reader answers the same question where an operator is already reading, and it answers it exactly: opening a mailbox asks mailbox/setup_status, which runs the Setup tab's own checks for that mailbox and returns its verdict. Anything the tab paints amber or red banners the reader in place of the first conversation, naming the offending check and linking to the tab. A mailbox that is all green shows nothing at all — silence is the normal state, so the banner means something when it appears.

Both surfaces run the same grouping code (mailbox_setup_scope.php), so the banner and the tab cannot disagree: mailbox_setup_scoped_rows() builds the Receiving/Forwarding groups and mailbox_setup_verdict() grades them. A check that could not run (unknown), one that is legitimately undecidable yet (info), and a capability nobody turned on (optional) are all silent — a verdict that flaps with a DNS hiccup gets ignored.

The checks cost DNS lookups and host probes, so the verdict is remembered per operator rather than re-resolved on every mailbox click. Freshness comes from writing it wherever the checks have genuinely just run: rendering the Setup tab stamps the verdict for the mailbox it just checked, so fixing a record there and going back to the mailbox clears the banner immediately — no waiting out a cache. The five-minute expiry is only the backstop for a mailbox nobody has looked at, the reader's Refresh control forces a re-run, and a reader left open in another tab re-asks when it regains focus. An unknown result never overwrites a real answer: one failed lookup should not make the banner flap.

It is admin-only (permission 5+) and scoped to mailboxes the caller can already see: members reading their own mail never receive a verdict, and the member mount has no Setup page to link to.

A badge is a navigation hint, not a verdict. It says go and look; the Setup tab re-runs everything live and is the only thing that claims a domain is correct or broken. That is why the copy reads needs attention rather than broken, and why nothing else in the platform reads these signals — they never gate sending, provisioning or cutover.

Three signals feed it, tiered by cost, assembled in plugins/mailbox/includes/mailbox_setup_hints.php:

• Only a required failure flags a domain. A missing DMARC record is graded recommended — real advice, but a domain receiving mail perfectly well should not wear a badge saying otherwise.
• A check that could not run is not a failure. An unanswered resolver would otherwise make every badge flap with the first DNS hiccup, and flapping badges get ignored. When nothing could be evaluated at all the previous verdict is left alone rather than overwritten with an absence of information.

echo "From: [email protected]
To: [email protected]
Subject: Test

Hello" | php plugins/mailbox/utils/inbound_email_handler.php [email protected]
echo $?   # 0 = success, 67 = unknown alias, 75 = temp failure

• mailbox_max_per_window (default 0, which disables the cap) — max non-deleted stored messages per domain inside the forwarding rate-limit window. A store above the cap is deferred, not dropped: the delivery is temp-failed (Postfix retry / webhook 503) so the sender redelivers once the window rolls, and it is logged once as store_capped.
• mailbox_relay_orphan_grace_days (default 30) — how long the relay pull holds recoverable-but-not-yet-storable mail on the relay before aging it out. A blob whose domain is disabled/unconfigured, or whose Fortress owner is not yet resolvable, is held (not deleted) so re-enabling the domain or restoring the grant lets the next pull store it; past the grace window it is dropped with a loud log. The held count surfaces on the relay health as "No mail held on relay".

The recipient half of that key is meaningful only where iem_recipient holds the plain routing address — inbound rows. On a sealed mailbox a composed row's recipient is ciphertext (see Sealed Vault), so the constraint cannot recognize a provider-filed copy of a locally-composed send. Every IMAP store path therefore dedups by Message-ID alone against the alias's outbound/draft rows before storing (ImapIngestor::storeMessage): on a hit the composer's copy adopts the IMAP locator and the folder membership, and no new row is stored. That includes a self-addressed send: its appearance outside the Sent-role folder is the delivered copy of a message the composer's row already holds, so it reconciles there and the row is marked iem_self_delivered (see Mail addressed to yourself below). Where a Sent-folder sighting promotes an already-stored inbound row to outbound on a sealed mailbox, the row's plaintext recipient becomes a sealing debt (iem_reseal_pending), paid in-window by the mailbox_promoted_reseal deferred-work consumer (PromotedRowRepair), which also retires an outbound duplicate of a composer's copy when one exists: the composer's copy adopts the locator, the duplicate is stripped of its remote binding (so the trash push can never relocate the provider's copy) and soft-deleted.

Mail addressed to yourself

A message sent from a hosted mailbox to that same mailbox is one row, not a Sent copy plus a delivered copy. MailboxSender writes the row at send time; when the message completes the trip out through MX and back in through Postfix, InboundEmailRouter::storeMessage() finds that row — by Message-ID scoped to the mailbox, since the (Message-ID, recipient, direction) key differs in direction by construction and is blind to a sealed row's ciphertext recipient — stamps iem_self_delivered on it, adopts the delivery's DKIM/SPF/DMARC verdicts where the row still holds placeholders, and stores nothing. Repeating the delivery reconciles again rather than forking a row. storeDirectMessage() applies the same rule, since Direct discovery can resolve a domain this deployment hosts.

iem_self_delivered is what the Inbox reads: the view is otherwise "not outbound", and the flag is the one exception, so the message lists once in the Inbox and once in Sent, as it does in any mail client. Opening the conversation shows one message.

A delivery stores its own row as usual when there is nothing to reconcile onto: mail from outside, a self-send composed in another client, a self-send whose Sent copy the member discarded (the delivery is then their only copy), and a matching row that belongs to a different mailbox.

Attachment & message storage

A stored push message is a lean record: the database holds the small, searchable parts — headers, the decoded text bodies (iem_body_plain / iem_body_html), and the attachment manifest — while every non-text MIME part (real attachments and inline cid: images) is extracted at ingest into its own private File. The bytes live in exactly one place — the File — so nothing is stored twice, and each attachment inherits the File layer's bucket offload, small-VPS drain, and gated serving for free. On the happy path no raw RFC822 is retained — except its header block, kept byte-for-byte in iem_raw_headers (capped at 64 KB, sealed like the body on a sealing mailbox): the wire truth (Received chain, Content-Type/charset, DKIM as sent) that no parsed column preserves, and what the reader's Show original renders for a lean record (specs/mailbox_show_original_coverage.md).

The manifest is the glue. Each ima_ row keeps the email-specific MIME metadata (filename, content-type, size, MIME section, encoding, content-id, inline flag) and, for file-backed rows, ima_fil_file_id pointing at its File. Dispatch everywhere keys on presence of ima_fil_file_id, not the transport:

Local bytes win. A reference is what the platform has when it does not have the bytes, so any path that turns up the message's raw bytes for a reference-backed row takes them and the row becomes file-backed (AttachmentByteCustody::adopt()). Concretely: a message ingested over IMAP keeps only a manifest; importing an archive that holds the same message deduplicates the message and adopts its attachment bytes; a live SMTP delivery (Postfix or webhook) that dedupes against an IMAP-fed row — a combined alias — hands its bytes over the same way from storeMessage's dedup return; and a Joinery Direct delivery does too (adoptParts() — Direct delivers decoded parts rather than a raw document, so its parts are matched by Content-ID or filename+type). The reverse order already keeps them — the ingester only adds a server locator and leaves the manifest alone — so connecting a mailbox, importing an archive, or receiving the same mail live reach the same end state in any order, and the user never has to know which to do first. Adoption streams each part to disk (never a whole-attachment string) and is a bonus on top of dedup, never a condition of it.

The upgrade only touches rows whose bytes genuinely live elsewhere: a row with no File on a message whose raw is stored locally is a section pointer into that raw, so its bytes are already local and copying them out would duplicate custody. A soft-deleted message is never upgraded, and a part that cannot be matched to a manifest row uniquely on both sides — by Content-ID, then MIME section plus type, then filename plus type — is skipped and logged rather than guessed at, because attaching the wrong bytes to a row is worse than leaving a working reference alone. Identity columns are never rewritten; ima_size_bytes becomes the decoded size, which is what it means on every file-backed row.

Attachment access has two doors, one rule each. The member download endpoint (/profile/mailbox/attachment) authorizes by mailbox grant for both backings: the viewer may access the alias of the attachment's message (MailboxViewer; a NULL-alias catch-all message is superadmin-only) — an attachment is exactly as private as its message, so every grantee of a mailbox, including permission-0 members and shared-mailbox teammates, downloads its attachments. The admin endpoint keeps the File-level posture for file-backed rows: each attachment File carries fil_private, so File::is_viewable() admits the file's owner (the single grantee of an individual mailbox; a shared or catch-all alias is owned by User::USER_SYSTEM) or any admin (≥ 5) — the same algorithm serve.php's /uploads/* path uses. No image variants are generated — attachments are served as their original.

Ingest is all-or-nothing per message. Every non-text part is minted as a File via File::createFromBytes() and linked in the manifest; the text bodies are extracted as today. If any File write fails (disk full — the pressure this design relieves), the message's Files are rolled back and it falls back to persisting the whole raw with a section-pointer manifest — the raw-storage shape below. The degradation chain is lean record → raw-to-disk → inline-in-DB; ingest never aborts, and the fallback logs a distinct INBOUND_ATTACHMENT_EXTRACTION_FAILED marker so an operator sees disk pressure.

Download streams the bytes by where they live (see the table): a file-backed row reads its File; remote fetches the one part from IMAP; a legacy/fallback row extracts it from the stored raw. Retrieval and streaming (original ima_filename, attachment disposition, nosniff) are the shared helpers in includes/attachment_retrieval.php, used by both download endpoints — each endpoint gates first with its own authorization posture, then retrieves. A .eml is never reassembled from these parts; the export serves the stored original or nothing.

Preview reads an attachment without opening it. Beside every attachment the platform can turn into text, the reader draws an eye button; it opens a modal containing the document's words as plain text and nothing else — no page is rendered, no markup becomes a document, no font is loaded, no URL is fetched. The endpoint is mailbox/attachment_text (POST attachment_id), gated on the same mailbox-grant rule as the download (a preview is exactly as private as the file), throttled at 30 per five minutes per IP because each one costs a subprocess — refusals count against the throttle too — and ceilinged by mailbox_preview_max_bytes (15 MB) from the attachment's recorded size before any byte is fetched or decrypted. Parsing itself is DocumentText, in a subprocess under the parser jail (the joinery-jail user: no network, no forking, no config, no key) — PDF, Word, Excel, PowerPoint, OpenDocument, EPUB, RTF, XML/SVG, forwarded .eml, calendar invites, the whole text family, and a name-and-size manifest for a .zip. Whether the button appears is decided from the declared type and the filename, because most real PDFs arrive declared application/octet-stream; that is a UI hint only, and the extractor re-sniffs the bytes regardless. A scanned document, an encrypted one, and one too large to read each get their own sentence rather than a shared failure line.

A picture gets the other kind of preview. There is no text inside a photo to pull out, so an image attachment opens as the picture itself — the one preview that really is decoded, by the browser's image decoder. It is offered because the alternative it replaces is worse: downloading the file and opening it on your own computer. The modal says so plainly rather than repeating the text preview's promise. The bytes come through the same gated download endpoint and are given an image type in the browser, so a sender's declared type never decides how the response is treated, and a file that is not really a picture simply fails to decode. SVG is deliberately not in this path: it is markup wearing an image's name, and it previews as text. Which kind of preview an attachment gets — text, image, or none — is MailboxService::previewKind(), and the reader carries it as preview_kind on each attachment. On a protected mailbox the bytes are sealed, so a locked vault answers {locked:true} and the modal offers the one-tap unlock. Nothing is cached: extracted text from a sealed message is exactly as sensitive as the sealed body, and re-reading costs milliseconds.

Forward re-attaches the original's parts in one manifest-driven loop dispatching per row: a file-backed row reads its File, remote fetches from IMAP, a legacy raw row extracts the section. An inline (cid:) part is re-embedded with its original Content-ID via EmailMessage::attachInlineData() so the forwarded HTML body's cid: references still resolve in the recipient's client; every other part attaches normally. The message is rebuilt fresh (forwarding re-signs DKIM/SRS), so byte-exact replay was never on the wire.

Inline images in the readers. MailboxService::resolveInlineImages() — the single cid: rewrite implementation, shared by the Mailbox Reader thread endpoint (ajax/mailbox_thread.php), the single-message detail page (admin_mailbox_message.php), and the native transport (withSignedTransport()) — resolves each cid:<id> reference in an HTML body to a short-lived signed URL (docs/file_signed_urls.md, 1-hour TTL for the web readers) for the manifest row whose ima_content_id matches <id> in that message only — a Content-ID can never reach another message's parts. Signed URLs are required, not optional: the body renders inside a sandboxed srcdoc iframe whose opaque origin attaches no cookies to subresource requests, and mailbox visibility is a grant decision (MailboxViewer) that File::is_viewable()'s owner-or-admin rule cannot express — so a session-gated /uploads URL can never authorize inline images for any reader. Minting is the authorization statement: the resolver runs only on messages the caller has already scope-checked (the viewer's grant scope, or the admin permission gate). A link that outlives its TTL renders broken until the message is reopened, which mints fresh ones. Unmatched cid: references are left as-is (broken). This applies to file-backed inline parts; a purely on-demand IMAP (remote) inline part is not resolved here.

Raw storage (fallback, legacy, and IMAP)

When a message is stored as a raw (the extraction fallback, or a legacy row), the heavy raw RFC822 lives in the cheapest durable store for its transport, not in the iem_raw_message column. A per-row storage descriptor (iem_raw_storage_driver) says where, and one accessor resolves it so callers are tier-blind:

This is deliberately not a general per-mailbox override. A hosted domain keeps exactly one answer, because its DNS-shaped guarantees cannot vary per mailbox.

InboundEmailAlias::security_level() / seals_content() are the resolver — own value when set, the domain's otherwise. Every content-sealing decision asks the alias; nothing else reads iea_security_level, and nothing else reimplements the inherit rule. Domain identity — DKIM, the protected sending identity, the DNS shape, the relay map — keeps asking the domain.

The level switch itself is the same either way, and so is the ceremony: raising or lowering a pulled-in mailbox runs the checklist, the server-side re-verification, the receipt card and the backlog sealing that a domain does, scoped to that one mailbox. There is no second ceremony.

• Ingest — every store path resolves one answer through InboundEmailRouter::resolveSealTarget(): the MAILBOX's posture decides, and mail with no mailbox asks the domain. A Standard mailbox stores plaintext even when its owner has a vault. A sealing mailbox that cannot produce a key declines the message rather than writing it in the clear — see A sealing mailbox always has someone to seal to below. This covers pushed mail (storeMessage), Direct (storeDirectMessage) and IMAP-polled mail (storeExtracted) alike. Composing is an ingress path too and asks the same resolver, through MailboxSender::sealTargetFor() — used by the send and by the draft autosave, so a draft and the message it becomes can never disagree about being sealed.
• Relay seal target — RelayMapExporter::sealTargetForAlias() seals to the owner's vault key (key_kind=user, producing Fortress pending-parse rows) only for a Fortress domain; every other posture seals to the ambient transport key, which Joinery opens at pull and re-seals per the domain's own level.
• Setup/health DNS shape — InboundEmailSetupCheck expects the inverted protected shape (SPF without the box, p=reject; aspf=s; adkim=s, DKIM matching the sealed key) for a domain whose ied_is_protected_identity flag is set, and for that domain only. The enforcement flag is the branching key, not the security level: the shape instructs the world to reject anything the sealed key did not sign, and MailboxDkimSigner signs with that key only once the flag is on, so prescribing it at the level would hand a Fortress domain without send protection a record set that rejects its own outgoing mail. The ceremony asks for the shape explicitly while it runs (dnsPlan($domain, true)), which is the one exception.

• InboundEmailMailboxGrant::sync_for_alias() refuses any change to a sealing mailbox that would leave other than one holder, or a holder without a vault. Every grant-writing surface goes through it — the alias editor, the combined mailbox editor, headless provisioning, the ceremony's inline fixes. The one write that does not is the user-delete cascade, so deleting the sole holder of a sealing mailbox is refused too, at the moment the deletion is attempted.
• Store time declines rather than downgrades. If a sealing mailbox somehow cannot produce a key, the message is not stored — and declining always means try again later, never a bounce: the Postfix pipe exits tempfail, the webhook answers 503, the IMAP feed leaves the message on the source and reports why it stopped, a Direct delivery is held with its parts, and the relay blob stays on the relay un-acked. Once the mailbox is repaired, the held mail flows in on its own.

Logged in but locked is the state a Private/Fortress user sees most often, so it is defined once for every surface: every surface shows cleartext metadata; every content action becomes a one-tap unlock prompt, and the original action resumes after unlock without re-navigation. A sealed or Fortress-pending row renders the neutral placeholder Sealed message (MailboxService::SEALED_PLACEHOLDER) — never a visible third state — while threading, unread, labels, folders, times, and sizes render normally.

• Web reader (mailbox_reader.js) — listThreads()/getThread() carry a top-level locked flag; opening a sealed thread or searching sealed mail shows an inline Unlock to read button that runs the shared platform ceremony (JoineryVaultLock, Sealed Vault § The lock chip) and re-runs the original request. Explicit lock lives on the platform lock chip; the reader listens for the platform's joinery:vault-locked / joinery:vault-unlocked events, so a lock or unlock from the chip re-seals or reveals content in place.
• Native /api/v1 — mailbox/thread_list, mailbox/thread, and mailbox/mailboxes return metadata plus locked (per-mailbox on the switcher, with each mailbox's security_level); mailbox/send returns locked: true instead of sending when a Fortress compose has no open window (MailboxLockedException); mailbox/thread_action (mark/star/delete) is cleartext metadata and keeps working while locked.

A mailbox seals when its posture says so — Private or Fortress, resolved per mailbox — and its single owner (the alias's one grantee — a shared or catch-all mailbox is never sealed) holds a Sealed Vault (docs/sealed_vault.md, the platform's per-user X25519 key hierarchy and unlock window). Mail is the vault's first consumer: it supplies its own AD row-binding convention and content, and reuses every generic vault mechanism — key hierarchy, unlock window, the File decrypt hook, the sealed-field model hook, rotation, revocation. See the Passkeys and Account Security docs for the sign-in and unlock ceremonies themselves; this section covers only mail's own participation.

What's sealed. InboundEmailRouter::storeMessage() resolves the owner's vault before the insert: a sealing row is written with empty content columns from the start (iem_sender / iem_subject / iem_body_plain / iem_body_html), then a fresh per-message DEK seals each field and the row is UPDATEd — no plaintext is ever written, even transiently, and the insert + seal UPDATE run as one DB transaction, so no empty-content row can ever survive a seal failure (a failed delivery tempfails and the MTA's retry re-inserts from a clean slate instead of hitting the dedup constraint). iem_sealed_key (the DEK, sealed to the owner's vault public key), iem_key_generation (0 = never sealed), iem_sealed_owner_user_id (whose vault, recorded at seal time — decryption resolves the owner from the row itself, so later grant or alias changes can never strand sealed mail), and iem_content_sealed (true once the UPDATE lands) mark the row. Attachments seal under the same DEK (InboundEmailRouter::extractAttachmentsToFiles()), AD-bound to their MIME part id — no chicken-and-egg with the manifest row's serial id, which doesn't exist yet at seal time — and each manifest row records its own ima_is_sealed: sealed state is a per-file fact about the stored bytes, never an inference from the message's flags. If attachment extraction fails for a sealed mailbox, the whole raw is preserved sealed (one AEAD blob under the same DEK, iem_raw_sealed = true, opened in-window by getRawMessage()) with a section-pointer manifest — the same durability a plaintext mailbox gets from the raw fallback, with nothing written in the clear. A composed outbound row (MailboxSender::storeOutboundRow()) seals identically (same one-transaction insert + seal), and also seals iem_recipient — an outbound row's recipient list is real content (who you emailed; stored untruncated — the column is text), unlike an inbound row's iem_recipient, which is the receiving alias address (routing metadata, never sealed regardless of the row's sealed state). Standard-tier mail (no vault) is unaffected: iem_content_sealed stays false and every column holds plaintext, exactly as before this package.

Who else it went to. Every row also carries the message's To and Cc lists in iem_to / iem_cc, sealed on every direction (a received message's other recipients are as much the owner's content as the people they wrote to). Each ingest path fills them from what it has — the parsed headers on the SMTP push and relay paths, $msg['headers'] on the IMAP-extracted path, the header part on Joinery Direct, the typed lists on a composed row — through one parser, MailAddressList, so the stored shape is the same whichever way the message arrived: "Ford, Tom" <[email protected]>, "Beltran, Luis" <[email protected]> (names always quoted, addresses always in angle brackets, entries joined by , ; a reader that respects double quotes can split on commas without splitting a name). An empty list is NULL. A row stored before the columns existed answers from its retained iem_raw_headers at read time (MailboxService::addressListsFor, parsed in memory, never written back); a row with neither answers ''.

Reading. InboundEmailMessage::$sealed_fields + decryptSealedField() / decryptSealedFieldStatic() are the Sealed Vault's generic model read hook: any $msg->get('iem_body_plain') on a loaded model decrypts automatically when the owner's window is open, and throws VaultLockedException when it isn't. MailboxService's raw SQL reads (listThreads(), getThread()) batch-decrypt through decryptSealedFieldStatic() directly (mirroring plugins/joinery_ai/includes/ModelQueryExecutor.php's raw-row hook), catching a locked vault into a [locked - unlock your vault to view] placeholder rather than an error. Attachments decrypt through InboundEmailMessage::openSealedAttachment() — the one opener, which dispatches over two sealed shapes and plaintext, in that order:

The opener is reached five ways, and every one of them goes through it rather than inspecting a flag itself — which is what stops any of them handing back a sealed container as a successful read: the generic File decrypt hook (File::registerDecryptHook(File::SOURCE_EMAIL_ATTACHMENT, …), registered by plugins/mailbox/includes/bootstrap.php, called by File::serve_from_path() between reading the on-disk ciphertext and streaming the response), the per-attachment download endpoints (includes/attachment_retrieval.php opens explicitly after File::read_bytes(), which bypasses the serve hook), a forward's re-attach path (MailboxSender::readOriginalPartBytes()), the forward message synthesis (InboundEmailRouter), and the unattended attachment digest (EmailAttachmentDigest, a defensive catch that should never meet sealed bytes). Each passes the File it already holds, purely to save a re-load — a caller that does not still gets plaintext, because the opener resolves the File from the row. A locked vault becomes a generic 423 Locked on the serve path and a clean "Unlock your vault" message on the download endpoints — never a raw error or leaked ciphertext. The admin single-message viewer (admin_mailbox_message.php) gates on the same key-possession rule as anyone else — a permission-10 admin (including via login-as) with no open window for the message's owner sees a [locked] placeholder, never real content; permission is not a bypass.

Bootstrap. plugins/mailbox/includes/bootstrap.php is mail's one-time-per-request wiring point, loaded lazily by VaultUnlock::loadConsumerBootstraps() from every code path that needs a consumer's hooks live (the File decrypt-hook resolution, the rotation ceremony, and window-close) — it registers the File decrypt hook and mail's VaultUnlock::onReseal() / VaultUnlock::onWipe() callbacks in one place.

Search. A sealed mailbox's content columns are ciphertext, unsearchable in SQL. plugins/mailbox/includes/MailboxIndex.php is a disposable, per-owner SQLite FTS5 index — sender, subject, both bodies, and attachment filenames (never attachment contents) — held only in /dev/shm (RAM-backed, never touches disk in the clear) for the lifetime of the unlock window. The table is contentless (content='') and positionless (detail=none), keyed by the message id as its rowid: a search returns ids and nothing else, so the text is never read back, and every query is single tokens ANDed, so positions are never consulted — the working copy is about an eighth of what storing both would cost (~105 MB for a 101k-message mailbox). `PRAGMA user_version carries MailboxIndex::FORMAT`; a working copy or restored blob of any other format fails to open and is rebuilt from the sealed rows, so a shape change is one rebuild per owner, never a migration. contentless_delete needs SQLite 3.43+, which the provisioning check checkSearchIndexEngine() verifies with the real DDL.

Folding is batched, checkpointed, and bounded (specs/mailbox_search_incremental_fold.md): every id is delete-then-insert (so re-folding is harmless), the high-water mark advances per completed batch, and fold() takes an optional deadline plus a non-blocking per-user flock (/dev/shm/mailfts_{uid}.lock) so exactly one fold runs at a time — a second caller searches what is already indexed and reports the backlog instead of contending. The search request folds at most MailboxService::SEARCH_FOLD_BUDGET_SECONDS; a backlog beyond that (a bulk import, a long-offline owner) drains in-window through the mailbox_fts_fold deferred-work consumer, and until it catches up the response carries search_indexing: {remaining, total} and the reader shows a non-blocking "still indexing" banner. A failed SQLite write stops the pass with the mark at the last contiguous success — the mark never advances past a message that is not actually in the index.

A fold that completed or processed refolds re-seals and persists the working copy as a private File (seal-after-fold; the sealed blob and its bookkeeping — high-water mark, blob coverage, sealed DEK — live in imi_inbound_mailbox_search_index); a fold mid-backlog persists every PERSIST_MIN_ADVANCE messages, so a window close costs at most one chunk of re-folding rather than a per-slice rewrite of a multi-hundred- megabyte blob. The blob records the mark it covers (imi_blob_high_water) and a restore resets the live mark to it, so a blob that lags the mark can never open a silent coverage gap. A fold that changed nothing persists nothing, so repeated searches over unchanged mail never rewrite the blob. Sealing and restoring stream path-to-path in the chunked v1.stream. secretstream format (VaultCrypto::sealFieldFile/openFieldFile over SealedBox::sealStreamFile/ openStreamFile), so memory stays proportional to a chunk — never to the mailbox — at any index size. Missing, stale, corrupt, or in any other sealed shape → rebuild() from the sealed message rows; the cache is never the source of truth. InboundMailboxSearchIndex::sweepWorkingCopies() is the passive-close safety net for a working copy the wipe callback missed (an idle APCu expiry, a worker recycle); it is declared as that class's $retention_policy and runs in the daily retention sweep, so worst case a copy lingers until the next sweep. MailboxService::listThreads()'s q path consults the viewer's index for whatever part of the scope the viewer holds a grant for — one mailbox or all of them (sealedIndexScope()) — and unions those hits with the plain Postgres tsvector search, which answers for the rest of the scope (another member's unsealed mailbox under an all-access view, unmatched mail) and simply never matches a sealed row's ciphertext — so its GIN index (iem_fulltext_unsealed_idx) is partial on unsealed rows, and every full-text query carries the same predicate (MailboxService::FULLTEXT_INDEX_PREDICATE) so the planner uses it. Locked surfaces as search_locked in the response, not a silent empty result, and the Postgres half still runs so unsealed hits show under the unlock prompt; a scope with no sealed content left needs no unlock and no index.

The index covers every stored message in the owner's mailboxes — trashed ones included, drafts excepted — and the read scope decides what a search returns: hits are intersected with the caller's scope, so a mailbox search never surfaces trashed mail and a Trash search finds it. One rule, in one place. The Inbox tab is not a search scope: a query typed there covers All Mail — archived and sent included — and the response carries search_scope: 'all_mail' so the reader labels the widening; explicit scopes (Trash, Spam, Drafts, a label) bound their own searches. Coverage cannot be narrowed by filtering the fold, because the high-water mark advances past every row a pass saw: a row the fold skipped is skipped permanently, and a rebuild runs the same query. Pruning follows the row's existence rather than a flag — MailboxIndex::enqueueRefold() queues an id whose row is about to go, and the refold pass re-inserts only if the message is still there. The queue also carries content that appears behind the mark: a pending-parse row folds as a no-op (its content fields do not exist yet) and parsePendingMessage() enqueues its refold when the fields land. Processed refolds leave the queue only after a persist has carried them (or when no blob exists), so a restore of an older blob can never revive a stale entry.

Key rotation and window close. Mail's VaultUnlock::onReseal() callback re-seals every message on the generation being drained (`iem_key_generation = old_key_generation` — the only generation the ceremony's old secret can open; idempotent, so a retry skips already-flipped rows), purges the FTS blob (sealed under the now-superseded key; the next unlock rebuilds it), and throws if any row failed — per the vault's re-seal contract, so the ceremony never retires a generation whose mail is still sealed to it. Its VaultUnlock::onWipe() callback clears the /dev/shm working copy on an explicit lock, a credential event, or lockAll() — the persisted sealed blob is untouched, so the next unlock restores it without a rebuild.

No sideways copies. The inbound log viewer (iel_inbound_email_logs) never carries subject or body — every write passes an empty subject, sender/recipient addresses are logged as routing metadata only. Content-derived AI processing (plugins/joinery_ai/pipeline_jobs/EmailSecurityScanJob.php) excludes sealed rows from its candidate pool outright: it runs unattended with no unlock window, so a sealed message is simply never a scan candidate, not a retried failure. LearnSpamFeedback already only trains from a message's raw RFC822, which a sealed message never retains — nothing further was needed there.

Pre-launch backfill. logic/backfill_seal_logic.php (an in-window, session-authenticated API action, mailbox/backfill_seal) converges a user's already-stored, not-yet-sealed mail to the sealed form once they set up a vault — one bounded batch per call, called repeatedly until done: true. It seals what the read path expects per direction: an outbound row's iem_recipient seals as content, an inbound row's stays plaintext routing metadata. A message still carrying its raw (a legacy fallback-stored row) re-splits its attachments into sealed Files and destroys the raw; an already-lean row (Files already extracted before the vault existed) has its content columns sealed while its existing attachment Files stay plaintext — safe, because every byte reader keys on the per-file ima_is_sealed flag, so those Files keep streaming as-is (the accepted pre-launch residual; there are no production users yet).

Per-page cost. The reader reads iem_inbound_email_messages by mailbox on every page load — the Inbox list, the rail's unread badge, the per-mailbox totals — and the message class declares two partial indexes for exactly those reads (live rows by mailbox with the archived/read/spam/direction columns the queries filter on, and trashed rows by mailbox). The queries are shaped to them: the Inbox filter is `iem_is_archived = false (the column is NOT NULL; IS NOT TRUE` is not a btree operator), and totals and unread are two queries so the totals run index-only. A page of threads opens only the content it shows: the sender of every message, the subject/body/summary of the newest per thread, and the HTML body only when the plain part yields no preview.

Provisioning. ext-sqlite3 (with FTS5 compiled in) backs MailboxIndex and has no fallback — without it, search on a sealed mailbox is simply unavailable (the reader surfaces this, not a 500). InboundEmailHealth::checkSearchIndexEngine() verifies both the extension and FTS5 support. The unlock window's own host-hardening facts (APCu apc.mmap_file_mask, swap, coredumps) are the vault's own VaultHealth check (includes/VaultHealth.php), not repeated here.

Mail that belongs to no mailbox

The catch-all accepts mail for addresses nobody created — postmaster@, a typo, an address a spammer guessed. That mail is stored with no alias, so it has no mailbox and therefore no mailbox owner.

On a sealing domain it seals to the domain's owner (ied_owner_usr_user_id, the same person whose vault seals the domain's DKIM key). It arrived for the domain, so the domain's owner is whose it is. One key covers every such address, and no mailbox has to be created per address.

InboundEmailMessage::sealOwnerUserId() is the single answer to "whose key does this message seal to", used by both delivery and the backlog pass so the two can never disagree. The fallback applies only to mail with no mailbox: a mailbox with no owner, or with several, still has no single key and stays unsealed until an operator fixes the mailbox. Sealing one of those to the domain owner would hand someone else's mail to a third party.

Two consequences worth knowing. Only the domain owner can read unmatched mail — other all-access admins still see the rows but cannot decrypt them. And a domain cannot sit at Private or Fortress without an owner who holds a vault: the protection ceremony makes it a required prerequisite, with an inline control for the acting admin to claim ownership.

Letting AI read a sealed domain's mail

The AI email features (triage, security scan, calendar extraction) cannot read mail that is encrypted at rest unless the owner is signed in with their vault open — and even then, only if the domain has been set to allow it.

ied_ai_processing_enabled is that switch. It is off by default, appears on the domain form only for Private and Fortress domains (at Standard the server already reads the mail, so there is nothing to consent to), and turning it on requires a recent identity confirmation. Turning it off never does — withdrawing consent must not be harder than giving it.

A second, separate answer — ied_ai_processing_consent — says how far the domain's mail may travel to be read by a model: local (hardware the operator controls), trusted (a vendor the operator has accepted), or cloud. It shows at every security level, Standard included, because where mail is sent to be read is a different question from whether it is sealed on disk: a stranger's message leaving the box for a hosted API is exactly what the setting refuses. It starts at local, loosening it needs the same identity confirmation, and a recipe that would need more than the domain allows is refused with a message naming this page.

With it off, saving a recipe pointed at a mailbox on that domain is refused, and the message names the domain and the setting. The refusal happens at save time rather than at run time, so the failure mode is an explanation rather than a recipe that silently does nothing.

What it buys, stated plainly: with it on, the server reads that domain's mail during an unlock window and sends it to the configured model host. Overnight processing remains impossible on a sealed domain — summaries appear shortly after the owner opens their mail, never before they arrive.

Parsing the backlog

Fortress mail that arrived while the owner was logged out is stored unparsed. DeferredIngest turns it into readable fields, and is registered as a deferred-work consumer, so the backlog drains wherever the owner is on the site with their vault open — not only when they open the mailbox. It parses newest first, so the most recent mail becomes readable first, and because the AI jobs skip unparsed mail and also take the newest first, the two never work against each other.

Outbound send protection

Encryption at rest protects reading stored mail while locked; outbound send protection protects the sending identity. A domain flagged ied_is_protected_identity is a protected sending identity: while no unlock window is open, no credential on the box can produce a DMARC-passing message with a From: header at that domain. The enforcement point is other people's mail servers applying the domain's published DMARC policy — infrastructure a compromised (even root) box does not control.

The invariant holds by closing every ambient send path:

• Sealed DKIM key, signed in-app. The domain's DKIM private key is generated in-session, sealed to the owner's vault public key (ied_dkim_sealed_key, a crypto_box_seal envelope — the same one message DEKs use), and stored in the database. The plaintext never touches disk and is never given to opendkim. At compose time, inside an unlock window, MailboxDkimSigner::resolveFor() unwraps it and PHPMailer signs with it as an in-memory string (DKIM_private_string), zeroized (sodium_memzero) as soon as the send returns. Core send code names no mailbox symbol: SmtpProvider and EmailSender read two callables the plugin registers on MailIdentityGuard at bootstrap (a protected-domain predicate and the DKIM signer resolver, memoized per request). A locked window makes the resolver throw, and the compose path prompts a one-tap unlock rather than sending unsigned.
• Protected compose submits through the box's own SMTP transport. A hosted alias on a protected domain resolves an SmtpProvider on the forwarding SMTP coordinates (OutboundTransport::forHostedAlias()), never the ambient platform provider — that transport is where the in-app signer runs, and the injected transport is what marks the send as the session-gated compose path. DMARC acceptance rides the strict-aligned DKIM signature alone; the domain's SPF excludes the box by design.
• Box out of SPF; strict alignment. The protected domain's SPF (`v=spf1 -all) does not authorize the box, and its DMARC is p=reject; aspf=s; adkim=s`. Strict alignment is load-bearing: it stops the box-authorizing forwarding subdomain from aligning the bare domain.
• SRS envelope on the forwarding subdomain. Alias forwarding runs while logged out, so its SRS envelope leaves from ied_forwarding_subdomain — strictly per-domain, always a subdomain of the protected domain (e.g. fwd.<domain>), set on the Protect page. Its SPF authorizes the box and its MX points back at the box (Postfix accepts it via the pgsql domain map), so forwarded mail passes SPF and delivery-failure notices (DSNs to the SRS envelope) route back to the router. The forwarded message's From: is the original sender's own domain, so forwarding never needs the user's identity. The SRS bounce notification sends from the platform's default identity — the one the ambient provider is verified for — so the notice itself is deliverable.
• Ambient senders refused. EmailSender::send() refuses any transactional (no injected transport) send from a protected From-domain; only the session-gated mailbox compose path (which injects a transport) may send as the identity.

Setup verification inverts for a protected domain. The Setup tab checks that SPF excludes the box, DMARC is strict, the published DKIM record matches the sealed key's public half (ied_dkim_public_dns, cleartext so it verifies while locked), the forwarding subdomain's SPF authorizes the box and its MX resolves to the box, and the domain is not relay-provider-verified. These are REQUIRED, so InboundEmailHealth gates on them and activation blocks until the whole shape — forwarding subdomain included — is published. One assembly (InboundEmailSetupCheck::protectedShapeResults()) feeds both the Setup tab and the ceremony's pre-activation verify, so they can never disagree.

Fortress is a two-sided promise, and send protection is the second side. Nobody can read your mail (arrival sealing), and nobody can send as you (send protection). Raising a domain to Fortress delivers the first half immediately and seals a DKIM key (mailbox_protect_seal_new_key()), defaulting ied_forwarding_subdomain to fwd.<domain>. A Fortress domain without send protection is not finished — it is one anyone can still impersonate — and both the raise receipt (one step left) and the domain.send_protection check row say so. That row is REQUIRED, so an unfinished domain reads attention.

Unfinished is a transit state, never a resting one. It cannot be made simultaneous with the raise — the switch needs published DNS and a vault unlock — so the interface declares the domain in progress rather than pretending either that it is done or that the remaining step is optional.

The step is not in the general setup path. It is the completion of the Fortress raise, and the raise is already the advanced, gated ceremony. The guided box carries a single Finish Fortress entry, gated on the relay being live and the domain's MX cut over — offering the sending half before mail arrives through the relay would ask an operator to finish what has not started — and it disappears the moment protection is on.

The cost is real and is stated where the offer is made: every interactive send needs an unlock, and automated mail must move to a Standard subdomain. Those are reasons not to choose Fortress for a domain, not reasons to run Fortress half-on.

Send protection has no page of its own — the Setup tab's Advanced section is its whole surface. includes/protect_identity.php owns the state transitions and nothing else; mailbox_protect_handle_action() runs any protect_* action posted to Setup and redirects back to the focused domain with a flash. The Sending identity box under Advanced holds the entire arc: what send protection buys and what it costs, the owner question when the raise could not guess, the publish step, the pre-flight verification, the switch, and afterwards the lifecycle (replace the key, switch over, cancel, turn off, and the return address behind a disclosure).

The ceremony opens on an explicit gesture (?protect_setup=1). Only inside it does dnsPlan($domain, true) prescribe the protected shape; the DNS records and the verification are not re-rendered anywhere else, and protectedShapeResults() remains the single assembly.

Ordering is load-bearing: no step may cause silent rejection. Publishing the strict records first tells the world to reject anything the sealed key did not sign, while the sealed key is signing nothing — so mail leaves, the provider accepts it, and the recipient discards it with no bounce anyone sees. The order is therefore:

1. Publish the DKIM record. Changes nothing; asks nobody to reject anything.
2. Start signing — protect_activate, gated by signingReadinessChecks() (the sealed key's record plus the forwarding subdomain) and a vault unlock. DNS is still ambient, so mail passes on either signature. The cost lands here as a visible refusal: a locked vault stops the send with a message.
3. Publish the strict SPF and DMARC. The signature they demand is already on every message.

The Setup tab is that ceremony's parent surface. Its Still to set up box lists what a Private or Fortress domain cannot be complete without — the vault, at Fortress the relay, and once those are in place the Finish Fortress step — and the ceremony page's breadcrumb and footer link return there. The box carries outstanding work only and does not render at all when there is none. Saving a domain at Fortress lands on Setup focused on that domain, not on the ceremony. The ceremony keeps its own page rather than becoming a Setup card because it holds destructive actions (rotate, disable, re-generate) that do not belong on a diagnostics surface.

Setup focuses either a mailbox (?alias_id=) or, for a domain that has no mailbox yet, the domain itself (?domain_id=) — a domain is registered before its first address, and all of this setup is domain-level, so the domain state renders the guided steps and the DNS publish box and skips the per-address checks. The picker lists a domain only while it has no enabled alias; once one exists the mailbox entry reaches the same guidance.

Rotation is staged. On an enforced domain, Rotate key seals a fresh key under the next selector (mailk{n}) into the pending columns (ied_dkim_pending_*) while the live key keeps signing; Verify & cut over swaps pending → live only after the pending selector's published DNS record matches, and Cancel rotation abandons the staged key. The live key is never overwritten or destroyed until its replacement is proven in DNS. A vault key rotation re-seals the DKIM keys — live and pending — alongside the message DEKs (the plugin's onReseal callback), for every protected-domain owner regardless of mailbox grants, on the same fail-loud contract.

Automated mail (lists, receipts, notifications) that must run around the clock lives on a dedicated non-protected sending subdomain (e.g. mail.<domain>), signed ambiently by provision_dkim.sh as usual. Under the bare domain's adkim=s that subdomain's key can never sign as the bare domain, so a locked box can send as list@mail.<domain> but never as you@<domain>.

Hardened ingest relay

A deployment runs one of three receive topologies:

• Colocated (the default, the cost floor): the MTA stack runs on the Joinery box itself, exactly as install_email.sh builds it. Zero extra infrastructure.
• Relay-fronted, self-hosted: a minimal, hardened, disposable VPS at the public MX fronts every hosted domain. It buys a hidden origin, edge-sealed ingest, and a shrunken main box.
• Relay-fronted, hosted fleet slot: the same relay stack, run by the platform operator as a shared fleet. The deployment enrolls for a slot, points its domains' MX at a per-tenant hostname, and gets the same edge-sealed ingest and hidden origin with zero extra infrastructure. See Hosted relay fleet.

The relay stack is tenancy-native, and a self-hosted relay is a fleet of one. Every tenant on a relay has its own spool directory and its own root-owned registry entry (Ed25519 public key, domain allowlist, shard-policy limits). A self-hosted relay is simply a relay with one tenant (slug main, allowlist *, created by the build); a fleet shard is one with a tenant per enrolment. One codebase, one code path — N=1 is the degenerate case.

Health is the only window into a relay. There is no door, so GET /relay/ping carries everything a person would have learned from a shell, under one rule: service state is not tenant data, and anything per-tenant (spool, Postfix message counts, the journal excerpt) is reported only when the relay has exactly one tenant. A root timer on the relay collects the privileged facts every thirty seconds; the listener, which never gains root, merges them with what it measures itself.

Once a relay fronts a deployment it is the MX for all that deployment's hosted domains (a mixed MX would leak the origin). The security level controls where mail is sealed, never where it is routed.

The sealing binary

provisioning/relay-sealer/ is a single static Go binary, prebuilt at publish into provisioning/bin/relay-sealer-<uname -m> and installed by provision_relay.sh to /opt/joinery-relay/relay-sealer. It replaces the PHP pipe on the MX path as the Postfix joinery transport (raw on stdin, ${recipient} ${sender} as argv). The same binary is the relay's map merge unit (relay-sealer merge-maps — see Map sync), its API listener (relay-serve), the root applier and collector behind that listener (apply-requests, collect-status), and its birth report (birth-report); the binary's README documents each mode. For each accepted message it:

• Looks up the recipient's public key + routing in the merged routing.json (no database). Every entry names its owning tenant; the tenant's block carries the spool directory, SRS secret, forward From identity, transport key, and the shard-policy limits (per-tenant forward rate limit and spool quota — over quota temp-fails, so senders queue instead of one tenant filling the disk).
• Seals the entire raw message with crypto_box_seal (libsodium wire format, SealedBox::openDek-compatible) to that public key — Fortress recipients to the owner's vault key, Standard/Private to the ambient transport key Joinery holds.
• Writes <spoolid>.seal (ciphertext) + <spoolid>.meta (cleartext operational metadata only — recipient, Message-ID, thread inputs, size, the milter-stamped Authentication-Results; never subject or body) via write-tempfile → fsync → atomic rename, returning the Postfix exit code only after the fsync. Plaintext is never written to the relay's disk.
• Executes forward-mode aliases relay-side, applying the identical header treatment InboundEmailRouter::buildForwardMessage applies (a byte-for-byte Go port with a parity test): rewrite From to the site's verified address so the original sender domain's DMARC never judges us, preserve the original sender as Reply-To, stamp the X-Forwarded-* headers, and SRS-rewrite the envelope sender (byte-compatible with SRSRewriter, so bounces decode on the main box).
• Stores SRS bounces: a delivery-failure notice returning to SRS0=…@forwardingdomain is accepted (a Postfix regexp map), transport-sealed, and spooled; the pull consumer routes it through the same handleSRSBounce path colocated ingest uses, so the original sender gets the NDR (never a stray stored message).

The relay-side merge (relay-sealer merge-maps, root, triggered by the path unit — never a resident daemon) is where the domain-claim boundary is mechanically enforced: every domain a fragment names must sit inside that tenant's root-owned allowlist (/opt/joinery-relay/tenants/<slug>/allowed_domains — * on a self-hosted fleet of one, the explicit TXT-verified list on a fleet shard), and must not be claimed by another tenant on the relay. A fragment violating either is rejected whole — nothing from it is installed, and the tenant's last accepted fragment keeps serving so a bad push never erases working routing. From the validated fragments the merge derives all the Postfix artifacts — relay_domains, check_recipient_access (preserving reject_unmatched: listed aliases match before a domain REJECT, so no backscatter), transport_maps, the SRS-bounce accept regexp map — plus the merged routing.json, installs atomically, and runs postmap + `postfix reload` only when the output changed. Shard-policy limits (tenants/<slug>/limits.json) are stamped into the merged tenant block here, so a fragment can never raise its own caps.

The tenant-side push is content-hashed and skipped when unchanged, and the verdict echoes the pushed fragment's version so the sync knows the merge saw this push. Every routing change (alias/domain/grant write, via a data-layer hook) triggers an immediate best-effort push, and the SyncRelayMap scheduled task reconciles every cron pass as the backstop — so a newly created alias reaches the relay before it can bounce.

The Setup tab's relay card grades map freshness accordingly (InboundEmailHealth::checkRelayMapFresh): a map that differs from the last push while the reconcile task is alive and succeeding (a run within the last 15 minutes) is pending — an amber wait ("address changes are queued"), because every domain/alias/system-sender change passes through that window for up to one tick. It is a red failure only when the task will not converge it: never ran, errored on its last run, or missed the window. Pending rides ProvisioningCheckPending, a subclass of ProvisioningCheckFailed, so every surface that cannot render the middle grade still treats it as unmet.

Spool pull + deferred ingest

The pull (RelaySpoolConsumer, phase 2 of the MailboxRelayReconcile scheduled task) dials out to the relay and copies new entries from its own spool directory (a signed GET /relay/spool listing and per-entry fetches), stores each durably keyed on the spool id (an idempotent re-pull is a no-op), and acks the entries it stored (POST /relay/spool/ack; ids only — the relay resolves them inside the tenant's spool and rejects anything with a path separator) — the delete-after-store is the ack. Standard/Private blobs are opened at pull with the ambient transport key and run through today's ingest. Fortress blobs cannot be opened while the owner is logged out, so they land as pending-parse rows: operational metadata + the sealed blob, so threading and unread counts work while subject/sender/body/attachments do not exist yet. At the next unlock, DeferredIngest unseals each blob, runs the full pipeline (parse, filters, attachment split, seal fields under a fresh per-message DEK), and clears the pending state. For a single reader this is invisible — the rules have always run by the time any mailbox view renders.

The reader's Refresh button runs the same pull on demand (mailbox/check_mail API action) before re-reading the list, so a user waiting on a message waits on one click, not on the cron interval. A per-relay advisory try-lock inside RelaySpoolConsumer::pull() keeps a click and the scheduled pass from racing (the loser reports skipped), and a short cooldown on the relay's last-pull time absorbs rapid clicking. On a direct-MX deployment the relay lane is a fast no-op — mail is pushed at SMTP time and there is nothing to pull.

check_mail covers the other pull lane too: it fetches the enabled IMAP feeds bound to the viewer's accessible mailboxes (ImapFetch::run — the same full cycle the scheduled poller runs), bypassing each account's poll interval. Most-starved accounts go first, a few per click, and an atomic 15-second claim on each account's last-poll time collapses rapid clicks to one fetch; the per-account advisory fetch lock inside ImapIngestor::poll() keeps a click and the scheduled poller off the same account at once.

A click is answered in bounded time. A browser, and the proxy in front of it, waits only so long for a request, so the IMAP lane hands every account one absolute deadline, ImapFetch::INTERACTIVE_BUDGET_SECONDS after the click. A cycle stops between folders and between messages once it passes — nothing already started is cut — with the folder cursor held below the first message not walked, so nothing is skipped, only left. An account the deadline stopped, or one the lane never reached, is left due (ImapFetch::leaveDue clears its last-poll stamp) so the scheduled poller finishes it at its next tick rather than a full interval later. The response reports each lane's took_ms and the IMAP lane's deferred count. The admin Fetch now action runs under the same budget and says when it stopped early.

Degradation is safe: relay down → senders' MTAs retry for days; the plane unable to reach the relay → the relay keeps spooling sealed blobs until the next pull. Neither loses mail.

Outbound sending

The relay is inbound-only by default: it accepts, verifies, seals, spools, and forwards inbound mail, and carries no compose sends. Compose sends leave through the deployment's configured outbound provider over an HTTP-API raw-message path. SMTP submission would stamp the main box IP into the sent message's first Received: header; an API submission's Received: chain begins inside the provider, so the origin stays hidden. OutboundTransport builds a fully formed, in-app-signed message (RawRelayComposeTransport) and hands it to the active provider's relayRawMessage() — the ApiSubmissionRelay capability (Mailgun's messages.mime, SES's Content.Raw). An API provider passes by construction; an SMTP provider carries hidden-origin sends only once the origin-leak probe has round-tripped clean within its window (InboundEmailHealth::hiddenOriginSendAllowed), and is refused otherwise with the probe named as the remedy — an operator running their own Postfix strips the submission Received: line, runs the probe, and sends. DKIM signing stays in-app: a protected domain signs with its vault-sealed key, a standard domain with the filesystem key opendkim would have used, and the envelope (MAIL FROM) routes through the forwarding subdomain so the protected domain's own v=spf1 -all never touches the envelope. Generated headers (Message-ID, etc.) derive from the mail hostname — which points at the relay — never gethostname() or the box IP.

Outbound confidentiality is bounded by the recipient's provider anyway: every message to an external address is delivered in plaintext to the recipient's mailbox provider, so a provider carrying it in transit adds a second reader to a set that already has one. Mail whose transit privacy genuinely matters is the encrypted-interop path, which is ciphertext before it leaves the box and stays ciphertext through any transport. The asymmetry lives on inbound, which lands in the operator's own archive under the user's keys — and inbound keeps the relay.

The relay never carries sent mail. It is inbound only: a tenant key can read its own spool and write its own routing fragment, nothing more, so a compromised relay can send nothing as this deployment.

The relay's outbound health checks are the provider's: checkOutboundTransportClass reads the same verdict as the send gate, and an out-and-back origin-leak probe is offered: sendOriginProbe() sends a marked message from the first enabled store-mode alias on a Standard or Private domain to itself — a listed alias because the relay's SMTP-time recipient validation rejects anything else, store-mode so the delivered copy lands in iem_inbound_email_messages, and non-Fortress so that copy is server-readable — out via the provider, back via the relay MX, and checkOutboundOriginLeak scans the delivered headers for the box IP or hostname on token boundaries. A clean probe within the freshness window (ORIGIN_PROBE_FRESH_DAYS) is what clears an SMTP path.

Provisioning

provisioning/provision_relay.sh is the self-contained build the relay runs on itself at first boot: idempotent, zero prompts, root on a fresh minimal Debian/Ubuntu image, driven by the first-boot user-data (provisioning/relay_first_boot.sh, rendered by the plane). It installs the prebuilt sealer from the bundle beside it, wires Postfix + opendkim (verify, RemoveARFrom stripping forged Authentication-Results) + opendmarc (stamp) + rspamd, creates the relay's identity, registers tenant main from the client public key in the user-data (or nothing, --skeleton-only, for a fleet shard with the operator's key), writes the three systemd units (joinery-relay-serve, the joinery-relay-apply path unit, the joinery-relay-collect timer), enables systemd-timesyncd, turns on unattended-upgrades' automatic reboot at a fixed hour, opens 25 and 443 and nothing else, and writes the signed birth report. rspamd is stateless: static rules only, Bayes classifier and autolearn off, no redis — learned state on a shared relay would be one model trained on every tenant's mail (a cross-tenant privacy leak and a poisoning vector), and the relay's header was never the verdict anyway — each tenant's own rspamd re-scores at ingest. --keep-sshd exists for one thing, a hand run on a box you can watch, and is refused without a terminal.

Tenant lifecycle on a relay is `relay-sealer tenant-add | tenant-set-domains | tenant-remove` — the same functions the operator-signed tenant routes call — so a tenant is a spool directory, a fragment drop and a registry entry (public key, allowlist, limits): no account, no peer, no sudoers rule. tenant-remove refuses while the tenant's spool holds undrained sealed mail.

The main box's half is provisioning/provision_relay_main.sh (root, once per deployment): it installs two narrow root helpers with their sudoers rule — joinery-mail-listener, the guarded local-listener switch, and joinery-dkim-remove. Nothing about a relay lives on the main box's disk: the relay client identity and the relay's pin are rows in the database.

The Setup tab's Relay section (rendered whenever the receive mode is relay or a relay row exists) is the dashboard: it lists each relay with its provisioning checks (reachable, spool draining, map fresh, origin hidden) plus the relay's last health answer with every group the relay reported behind a disclosure, and its guided controls create, update, enable/disable, delete, and Check spam scanning now.

Is the relay still scanning?

Everything else the relay does leaves evidence in the tenant's database. opendkim and opendmarc stamp a verdict onto every message, so a broken verifier shows up as unverified mail. rspamd does not: it stamps a header only when it flags something, and milter_default_action = accept means a dead scanner lets mail through rather than deferring it. A relay that scanned and found nothing and a relay whose scanner is dead therefore send identical evidence — none — and no amount of reading stored mail can separate them. Warning on "no message carried a content verdict in N days" reports every quiet mailbox behind a healthy relay as broken.

So the relay is asked. GET /relay/ping (signed, pinned) answers one JSON object whose first keys are the scanner verdict:

{"status":"ok",
 "services":{"rspamd":"active","opendkim":"active","opendmarc":"active","postfix":"active",
             "joinery-relay-serve":"active","joinery-relay-apply.path":"active","joinery-relay-collect.timer":"active"},
 "milters":{"opendkim":true,"opendmarc":true,"rspamd":true},
 "contract":true,"provisioned":"3.0","slug":"main","sole":true,"queue":0,
 "build":{}, "identity":{}, "service_detail":{}, "listeners":{}, "tls":{}, "clock":{},
 "machine":{}, "firewall":[], "postfix":{}, "spool":{}, "direct":{}, "auth":{}, "log":[]}

services is unit state from the relay's root collector (a thirty-second timer; before its first pass every unit reads unknown, never active); milters is read from postconf -h smtpd_milters; contract is the header contract — the build writes local.d/milter_headers.conf and local.d/actions.conf itself and records their digest in /opt/joinery-relay/contract.sha256, so the collector re-hashes and returns a boolean rather than PHP modelling rspamd's config format. A relay whose contract drifted scans perfectly and stamps nothing InboundEmailRouter::readSpamHeader() can parse, which is why service liveness alone is not the question. The remaining groups are everything else a person would have learned from a shell — build, identity, listeners, certificate, clock, machine, firewall rule set, Postfix counts, spool, Direct, signature failures, the relay's own journal — and the Setup tab renders them behind a disclosure, because health is the only window into a relay.

Service liveness only, per tenant where it is the asker's. A shared fleet shard serves several deployments, so the answer carries queue depth, message counts, spool sizes and the journal excerpt only when the relay has exactly one tenant; on a shard those keys are absent, never zero. Service state is not tenant data.

MailboxRelay::readHealth() turns one answer into a state (ok, not_delivering, legacy, unreadable, unreachable) plus a reason (dead, unwired, drift). pollHealth() runs the ping and caches the answer on the relay row (mrl_last_health_json / mrl_last_health_time); an unreachable result is deliberately not cached, because overwriting the last real answer destroys the only information available during an outage.

MailboxRelayReconcile polls once per pass, and InboundEmailSetupCheck::checkRelayScannerHealth() reads the cached answer, so no page render pays for a round trip. Check spam scanning now in the Relay section forces a fresh one for an operator mid-incident. Severity depends on whether this server is covering:

A relay keeps accepting mail while this server cannot collect it: senders are told their message was delivered, and it sits sealed on the relay's spool until something pulls it. So the reconcile pass treats a pull that cannot reach the relay as an outage, not as "does not apply" — a relay row with no identity pin answers error from both the spool pull and the map push — and announces the outage itself rather than waiting to be found:

• mailbox.relay_pickup_stopped is raised once when a pull fails and no pull has reached the relay for MailboxRelay::PICKUP_STALL_SECONDS (30 minutes, the same threshold as the Setup tab's Mail pickup check), or ever. The pass stamps mrl_pickup_alarm_time so a relay that stays broken is not re-announced. The notification says what stopped, when mail was last collected, that senders are being told it was delivered, and what to open.
• mailbox.relay_pickup_recovered is raised once when a pull reaches the relay again while the alarm stands — an empty spool counts, because reaching the relay is the fact — and the stamp is cleared.
• MailboxRelay::pickupTransition() is the pure rule both come from; a pull that was skipped (another pull running, no address yet) says nothing either way.

• Every admin page carries MailboxAttentionNotice through the admin- header registry (AdminNotices, docs/admin_pages.md § Admin header notices): Mail is not arriving while the alarm stands, This server cannot reach the relay while the last ping (mrl_last_health_failure) failed, and nothing otherwise. It never pings or probes.
• The reader's setup banner (This mailbox needs attention) is the Setup tab's verdict for the open mailbox, and the relay's Receiving card is part of it for a Fortress domain. Both reader mounts show it to an operator: the admin reader always, and the profile reader (/profile/mailbox/mailbox) for a viewer at permission 5 or above, since that is where operators read mail too. Members never see it.

With every domain fronted, the relay is the sole mail listener: the main box's Postfix/opendkim/opendmarc are decommissioned and port 25 closed — the box holding the data no longer exposes a mail listener. rspamd stays where it was running: the scanner ships with the mail stack, so it is on every box that ever ran the mail installer, and the decommission leaves it alone — a learning deployment simply switches it from milter mode to scoring pulled mail over HTTP at ingest, carrying its Bayes corpus across the move with no reinstall. The relay scores regardless (provision_relay.sh installs rspamd unconditionally, stateless) and stamps its X-Spam header inside the sealed raw. The setup/health checks retarget to the relay (checkRelayReachable, checkRelaySpoolDraining, checkRelayMapFresh) and add a deployment-wide origin-hidden check (checkOriginHidden) that fails if the main box IP appears in any hosted domain's mail DNS.

Decommission is a guarded platform action, never manual host surgery (includes/listener_admin.php). The Setup tab's Relay section shows the offer only when it is actually possible: once every guardrail passes, an amber Uninstall local mail block appears (one sentence — the relay makes the local mail software unnecessary and a security risk — plus the button); while any guardrail fails, nothing renders at all (the Setup rows already walk the missing pieces), and the server-side re-check on POST remains the enforcement. The button runs /usr/local/sbin/joinery-mail-listener off — a narrow root helper installed by provision_relay_main.sh — which stops and disables Postfix/opendkim/opendmarc and closes 25/tcp at the firewall. After an uninstall the block goes quiet while a relay is still receiving mail — reinstalling would reopen attack surface no mail would use — and returns only once no enabled relay remains, as an amber warning that this server has no way left to receive mail plus Reinstall local mail (on, the always-safe inverse; it has no guardrails of its own). The one exception is a setting/reality mismatch: if port 25 answers while the record says uninstalled, the red block appears regardless of the relay and re-offers the uninstall. The guardrails: an enabled relay exists, DNS has fully cut over (InboundEmailSetupCheck::relayCutoverState(), the same evaluation behind the cutover-completion row), the spool pull is healthy (checkRelaySpoolDraining), and outbound does not lean on the local Postfix (no provider, or SMTP aimed at localhost). The outcome is recorded, not inferred: the mailbox_local_listener setting (active | decommissioned) is written only on a successful helper run, and the host.port25 / host.postfix / host.opendkim setup rows and InboundEmailHealth::checkInboundMailServer() compare it with reality — under decommissioned, an answering port 25 is the failure, and silence is the healthy state. The helper runs on the deployment's own box via a sudoers rule, so standalone tenants get the same button with no server_manager dependency.

Hosted relay fleet

The platform operator runs a shared fleet of hardened relays as a service (specs/mailbox_relay_shared_fleet.md). A shard is exactly the self-hosted relay stack fronting many tenants; a slot is one tenant deployment's place on a shard. The trust statement is published plainly: the fleet operator stands at the plaintext-arrival moment for inbound transit mail and could read it while actively compromised — the same position any hosted MX occupies. It can never reach the tenant's archive, keys, drive, passwords, or sending identity (DKIM keys never leave the tenant's app). The exit ramp: point your MX at your own relay whenever you want — same stack, nothing else changes (fleet_release). Release revokes the slot's domain claims immediately, so the domains' next home (a new slot here or another fleet) can claim them before the old slot finishes evicting.

Tenant side (any deployment): every tenant-facing hosted-relay surface — the Setup Relay section's Hosted relay block, the Settings connection box, and the live fleet-status fetch — is gated behind mailbox_hosted_relay_offered() (includes/receive_mode.php), which is off: the hosted offering is not customer-facing yet, and the choice card/Relay section describe only the run-your-own path. The fleet API actions and the operator console are unaffected. When offered, the Settings tab's Hosted relay connection box takes the operator's service URL + the customer account's API key (mailbox_fleet_service_url / mailbox_fleet_api_public_key / mailbox_fleet_api_secret_key, a declared sealed secret stored encrypted wherever the box has a secret_box_key); enrollment itself is a button in the Setup tab's Relay section. FleetClient is the core ServiceClient — the one tenant-side shape for any service another deployment runs (an API key pair in public-key / secret-key headers, a JSON POST to /api/v1/action/{plugin}/{action}, the remote error text on refusal) — with the relay's own actions on top. It calls the operator's /api/v1/action/mailbox/fleet_* actions: fleet_enroll sends this box's relay client public key and returns the slot coordinates (per-tenant MX hostname, the shard's identity pin and address, spool directory), which fold into the deployment's MailboxRelay row (mrl_is_hosted) — after which every relay consumer runs exactly as against a self-hosted relay. Hosted vs self-hosted differs only in where the coordinates came from. Each domain must pass a DNS TXT ownership proof (_joinery-fleet-challenge.<domain>) before the fleet accepts a single message for it, with fleet-wide uniqueness; verification writes the domain into the tenant's shard-side allowlist, which the map merge enforces on every subsequent sync. The proof is fully automated on the tenant side: challenges are filed at enrollment and at domain registration (FleetClient::fileDomainClaims()), the Setup tab's domain.ownership row shows the copy-ready TXT record and re-verifies on every check pass, and the Relay section shows a read-only Ownership proofs state table. The fleet_claim_domain / fleet_verify_domain API actions are what that automation calls — they are not user-facing steps.

Operator side (the deployment with mailbox_fleet_service_enabled + mailbox_fleet_mx_zone set): the fleet service is the brain — FleetService assigns shards (least-loaded born shard with capacity), issues and verifies domain claims, and checks entitlement (the mailbox_fleet_slot tier feature). The relay reconcile walks each slot down the core ServiceTenantLadder — the grace-then-suspend ladder every rented service shares: a lapse starts a mailbox_fleet_grace_days window, suspension past it empties the tenant's shard allowlist, and a subscription that returns at any point reactivates the slot in place. The slot's provisioning | active | suspended | released states are the ladder's vocabulary; evicted is the fleet's own last rung. Every decision is effected as one operator-signed request to the shard's tenant routes (FleetService::applyTenant) from the relay reconcile task, the slot's status moving on the verdict. The operator's control panel is the relay fleet console (/plugins/mailbox/admin/admin_mailbox_fleet, reached from the Server Manager dashboard — operator infrastructure, so it never appears in the tenant mailbox tabs): the service switch + MX zone, shard registration (a skeleton-only birth in the operator's cloud account: the operator's box is not a tenant of its own shards), and the DNS-to-publish table. Each tenant's MX hostname (<slug>.<mailbox_fleet_mx_zone>, slug format t<id> — deliberately anonymous so DNS names no tenant) is an operator-controlled A record, so re-sharding a tenant or replacing a burned shard is an A-record change — tenants never touch DNS after setup. The operator's half of that guidance is the fleet console's DNS to publish table: every record the fleet zone needs — each shard's A record and PTR expectation, and one A record per live slot MX hostname — with a live resolution verdict and copy fields. (PTR records are set where the shard's IP is hosted, not in the DNS zone.)

Selling slots — order-time auto-enrollment. The fleet console's Fortress hosting product box creates the sellable product in one click (store + server_manager required): a subscription tier whose features grant mailbox_fleet_slot (an existing slot-granting tier is reused) and an inactive customer_cloud-fulfilled product on it — pricing and activating it are the operator's explicit acts on the product edit page. When a paid order then provisions the buyer's server, ProvisionCustomerCloud hands the finished site to FleetProvisionSeeding (mailbox side) once its agent has paired: if the fleet service is on, the store is this deployment, and the buyer's tier carries the slot feature, it mints a machine API key for the buyer's account (Fleet enrollment, read+write; re-minting deactivates the previous one) and dispatches ONE fleet_enroll job on the site's own agent carrying the three values — the setting names are compiled into utils/fleet_enroll.php on the site, the secret rides the job row redacted and is blanked once the node answers, and nothing opens a shell. The owner's Setup tab then lands on one-click Enroll; the DNS TXT ownership proofs and the MX edit stay manual by nature (the customer proving domain control at their own DNS provider). Seeding is best-effort: a failure alerts the ops address and leaves the provision done — the owner can always enter the credentials manually on the Settings tab. A subscription carries one slot, so one seeded site per buyer: seeding a second site is refused, naming the site that holds the slot (re-seeding that same site is a key rotation). A site that has been decommissioned holds nothing — its node row is soft-deleted — so the slot is free for the account's next site.

Rebuild carries the spool across the wipe. The scheduled shard rebuild closes port 25, flushes the Postfix queue for a bounded window, copies the per-tenant spools and any still-deferred queue files aside, re-runs the full provisioning, and restores with a validating pass (strict <id>.seal / <id>.meta name pattern, owning tenant's directory, correct ownership, no exec bits) before reopening 25 — so no accepted message is ever lost in a rebuild; mail not yet accepted waits at senders' MTAs. Self-hosted rebuilds use the same sequence; N=1 is the same job.

Mailbox Reader

The Mailbox Reader is a two-pane Gmail-style reader over the stored messages: a left rail (the mailbox switcher and its folders) and a single main pane that shows either the conversation list or an opened conversation full-width. It is a vanilla-JS client (assets/mailbox_reader.js + .css, cache-busted by file mtime) talking to the scoped AJAX/API actions listed under API Surface.

The reader has two mounts of one shared UI (includes/mailbox_reader_mount.php):

• Admin — the Mailboxes tab (admin_mailbox_reader.php), staff chrome, with attachment downloads at the admin endpoint.
• Member — /profile/mailbox/mailbox (views/profile/mailbox.php + logic/profile_mailbox_logic.php), theme chrome, for any signed-in member; what they see is their granted mailboxes. A member with no grants gets a short "no mailboxes are assigned to your account" state. Attachment chips point at the member endpoint. The plugin's profileMenu declares the "Email" entry that puts the page in the member menu on every theme and in the apps' navigation. A gear at the right of the page header holds what the mailbox needs that is neither reading nor writing mail: Filters (/profile/mailbox/filters), Email settings (/profile/mailbox/settings) and Import old mail (/profile/mailbox/import).

Selecting rows in the list and acting on them sends the whole selection to thread_action as thread_keys[], which expands each key under the caller's own scope and unions the resulting message ids. A key the caller cannot see contributes nothing, so naming a conversation can never reach it — the same guarantee the single-thread_key path carries, and the mutations re-check scope in SQL besides.

A debounced search box sits in the conversation-list header. It runs a single PostgreSQL full-text query (websearch_to_tsquery) over the sender, subject, and both plain and HTML body fields at once, backed by the iem_fulltext_idx GIN index on the matching to_tsvector expression. Searching the HTML body directly is safe: PostgreSQL's text-search parser classifies markup as tag and skips it, so an embedded stylesheet contributes no lexemes and an <a href> indexes only its link text. The expression lives once in MailboxService::FULLTEXT_SQL and the migration builds the index from that constant — the index serves the query only while the two match byte for byte, and a silently-unused index is indistinguishable from a slow one. Each column in the expression is left()-capped (250 KB per body) because a tsvector has a hard 1 MiB limit and the index is evaluated on INSERT — uncapped, a message with more text than that is not merely unsearchable but unstorable, failing the insert itself. A mailbox whose IMAP feed has discovered folders also lists them indented under the selected mailbox (an "All Mail" root for the folder-unfiltered view, then each tracked folder); see the Sync subsection for how membership drives folder contents.

Threading and shared state

Threading is by iem_thread_key, computed at store time by InboundEmailRouter::computeThreadKey() (References first token → In-Reply-To → own Message-ID → null; a null key is a singleton, keyed client-side as m:<id>). Subject-based grouping for header-less mail is a deliberate non-goal.

The row's preview is the plain body when there is one, and otherwise the reading text of the HTML via MailboxHtmlSanitizer::toReadableText(). That is a DOM walk, not a strip_tags(): received bulk mail carries its stylesheet inside the document, and stripping tags keeps the CSS between them, so the preview would read a.cta_button{-moz-box-sizing…. The walk runs in the parser jail (DocumentText::parseWithMany(), one subprocess for the whole page of rows), never in the request: received HTML is a stranger's bytes and libxml2 is a C parser. The walk drops <style>, <script>, <head>, <title> and comments with their contents, treats block edges as word boundaries (table-built mail otherwise reads as benefitTerms apply), keeps link text without the URL, and removes the invisible characters — zero-width joiners, soft hyphens, combining grapheme joiners, non-breaking spaces — that senders use to pad a preheader. An image-only message previews as empty, which is honest. toPlainText() remains the separate, faithful plaintext copy of mail we composed, and does render links as text <url>.

The sealed search index reduces an HTML body the same way, so a sender's stylesheet is never searchable — otherwise a search for container or sans-serif matches every newsletter in the mailbox. Changing what MailboxIndex::rowContent() indexes changes what a stored index holds: purgePersisted() the affected owners so the next unlock rebuilds, or the old text keeps matching.

A list row also carries a paperclip between the subject line and the time when any message in the thread has a real attachment — has_attachment on the thread payload, from one id-only query over the page's messages (MailboxService::messageIdsWithAttachments()). Inline cid: parts do not count: they are body content, not something the reader would go looking for, so a message whose only "attachment" is an embedded signature image shows no clip.

Read/star state lives on the message row (iem_is_read, iem_is_starred, iem_read_time) — not in a per-viewer table. On a shared mailbox this means read state is shared among everyone with access (team-inbox semantics: you see what a colleague already handled). Opening a thread marks it read for everyone on that mailbox. Read/star state is a property of the mailbox row, shared by everyone granted access to it.

Sender names

iem_sender holds the From display name beside the address, in the form "Name" <addr>. InboundEmailRouter::senderDisplayString() builds it for the Postfix and webhook paths (both the immediate store and the deferred parse a sealed mailbox uses); the IMAP and archive paths get the same shape from Horde's envelope. Encoded words are decoded, a name identical to the address is dropped, and the name — attacker-chosen text arriving over SMTP — is stripped of quotes, angle brackets and CR/LF before being quoted. When the column limit bites, the name is what gets cut: an address short of bytes is an unreplyable sender.

The From addr-spec is the last angle-addr in the header, not the first. A display name may legally be a quoted string containing angle brackets, so From: "Support <[email protected]>" <[email protected]> is a valid header whose real address is [email protected]. Reading the first one would let a sender choose the address used for iem_sender, the reply address, the contact lookup, filter matching and the SRS envelope, while the authentication results still described the domain that actually sent the message.

The reader's list column shows the name and keeps the address on the row's hover title. With no display name at all, the sending organization is the label — [email protected] reads as Fireworks, since the local part of automated mail (no-reply, meet, product) identifies nothing. The organization label is the last host label below the public suffix, which drops infrastructure subdomains for free (accounts.google.com → Google). The exception is a consumer mail provider, where the person is the only identity available, so the local part is used instead ([email protected] → Jeremy Tunnell, never Gmail).

That exception covers only what could actually be somebody's mailbox. A role address — no-reply@, support@, or anything carrying a no-reply marker (AmericanExpress-no-reply) — is infrastructure, and an address below a provider's own domain is the provider writing rather than one of its users, since a personal mailbox never lives at a subdomain. Both fall back to the organization: [email protected] reads as Proton, not No-Reply.

All four lists live at the top of mailbox_reader.js. The same rules are mirrored in the native mail kits (MailDisplay in ios/joinery-kit/.../MailModels.swift and android/joinery-android-mail/.../MailModels.kt) so one message reads the same in the app and the browser — change them together. Each of the three has its own guard: plugins/mailbox/tests/sender_name.mjs, MailParsingTests.swift, and MailParsingTest.kt.

An open message shows the name and the address (senderFull()). A display name is only ever as trustworthy as the domain behind it, and the domain is the part that survived DKIM.

The viewer seam

MailboxViewer (includes/MailboxViewer.php) answers who is looking and what may they touch:

• accessibleAliasIds() — for a permission-10 superadmin, every alias (all-access oversight: every mailbox plus a merged "All mail" view that also surfaces unmatched NULL-alias mail); otherwise the aliases the viewer holds a grant for.
• scopeAliasIds(?int $aliasId) — the single place audience becomes a query filter: an accessible alias → [id]; a null selection → the full accessible set; a non-accessible alias → [] (matches nothing). The superadmin "All mail" unconstrained case is handled in MailboxService, gated by isAllAccess().

Rows are sealed when that user holds a vault (imc_address / imc_display_name under a per-row DEK); dedup is imc_address_hash — a keyed blind index for vault holders (never leaks the sealed address), plain SHA-256 otherwise. The hash covers the mailbox and the address together, which is what makes the existing (hash, user) unique constraint mean one row per (user, mailbox, address) without a composite key over an encrypted column.

The composer fetches the whole (small) decrypted list for one mailbox and filters it client-side for To/Cc/Bcc autocomplete (no server prefix-search over ciphertext); a locked vault makes autocomplete silently absent. Changing the From selector re-fetches the list for the newly chosen mailbox, so suggestions always follow the address being written from. Addresses already typed are left alone — only the suggestion list changes.

Because nothing files itself, a row's mere presence means the user put it there — there is no seen-vs-saved distinction to report. MailboxContacts::lookup() returns how the row got there (manual or import) and when. Adding an address already held bumps the existing row rather than inserting a second: a hand-add re-stamps an imported row manual and fills a display name the import never carried. An add that cannot be written — a sealed store whose vault window has closed has nowhere to put the address — returns false, so the reader reports a failed add instead of appearing to have saved it.

The right column

The aside on the right is a stack of panels, one per slot: #mbx-context-ai, which a plugin docks its own panel into (the joinery_ai panel, when that plugin is active), and #mbx-context-people, the reader's own contact panel below it. The column shows while either slot holds something and narrows to a labelled spine only when every panel in it is collapsed, so one open panel keeps the column its full width. It is not shown below 1100px at all.

A docked panel and the column know two things about each other and nothing else: the panel marks its root data-collapsed and fires a bubbling joinerypanelcontent event when what it holds changes; the column answers by setting data-spine on each slot when it has narrowed. That is the whole contract — see plugins/joinery_ai/docs/overview.md § The area AI panel.

Contact panel

The reader's own panel in that column is where contacts live — the left rail lists where mail lives, and a contact store belongs to a mailbox rather than sitting beside one. The panel has two states over the same element:

• On the list view — the selected mailbox's contact manager (add, delete, and import a vCard / Google CSV via mailbox/contacts_import, all landing in that mailbox). Collapsed by default, since it is reference material rather than the task at hand; the open/closed choice is remembered across visits. A view with no one mailbox behind it (All mail, or an unmatched box) has no single store to show, so the panel steps aside entirely.
• On an open conversation — the correspondent's card, expanded (mailbox/sender_context). The client sends the message id, never an address, so the endpoint can't be a membership oracle: the server re-derives the counterparty from a message already in the caller's scope, and that scope binds admin and non-admin alike.

Spam is a first-class verdict on the message, iem_spam_verdict (ham / spam; NULL = not evaluated). It is what the reader filters on, so one Spam view works identically for locally-received mail and IMAP-polled mailboxes. There is no folder membership — the verdict is the disposition. The app runs no scorer of its own: it acts on the auth verdicts and on a binary spam result a content scanner (or the webhook provider) decides, recording the scanner's numeric score only for display. The one exception is SendGrid, which exposes a score but no binary, so its result is derived from a configurable threshold (see Content scanner).

Three protection layers stack: the MTA's RBLs at RCPT time, the auth rule below (DMARC/SPF/DKIM), and a content scanner — the only layer that catches authenticated bulk spam (junk that passes its own DMARC/SPF/DKIM: lookalike domains, bulk mail from real ESPs, a compromised aligned account). All three feed the same iem_spam_verdict.

Gated by mailbox_spam_filtering_enabled (default on), toggled on the Settings tab as Move suspected spam to the Spam view. When off, the verdict stays NULL and nothing changes. Default-on is safe because the disposition is reviewable — spam is moved, never rejected, bounced, deleted, or forwarded — and because the auth verdicts it acts on are recorded for every message regardless.

Classification rule. The router acts on the SPF/DKIM/DMARC verdicts it already records (it never computes them — see Inbound authentication). InboundEmailMessage::authRuleSaysSpam() holds the rule itself, and InboundEmailRouter::classifySpam() applies it alongside the content signal. One definition, because the reader asks the same question of a stored row when it explains why a message is in Spam:

• DMARC fail → spam. The primary rule. DMARC is alignment-based and already subsumes SPF and DKIM, so it is the one signal worth acting on directly. Applies wherever a DMARC verdict exists (Postfix milters, SES).
• No DMARC verdict, and SPF and DKIM both fail → spam. The fallback for providers that supply SPF/DKIM but no DMARC field (Mailgun, SendGrid). Both must fail: raw SPF/DKIM lack DMARC's alignment check, so a single failure has too many legitimate causes (forwarding breaks SPF; some legit mail breaks DKIM), whereas both failing is a clean "even basic auth broke" signal.
• otherwise ham.

Always allow a sender. A message filed by the auth rule carries a banner in the Spam view naming the reason and offering Always allow <address>. The button writes an explicit never_spam filter scoped to that mailbox, flagged for the "also apply to existing" backfill so mail already in Spam from that sender is swept up, and clears the message in hand immediately. This is the only route past an authentication failure, and it is deliberate, visible on the Filters page, and reversible by deleting the rule. A message filed on the content score gets no banner: Not spam teaches the scanner and adding the sender to contacts elevates them from then on. See specs/mailbox_contact_spam_bypass.md.

Forward suppression. A judged-spam message is never relayed — forwarding spam burns the platform's sending reputation and can relay abuse. The forward is suppressed and logged with status spam_held. A forward_and_store alias still stores the message (with its spam verdict) so it stays reviewable; only the outbound forward is dropped. Pure-store and catch-all-store aliases store as usual, verdict and all.

IMAP-polled mail. No auth rule runs — the remote server already classified it. A message ingested into a folder whose iif_role is junk is marked iem_spam_verdict = 'spam', giving the Spam view the same meaning for polled mail.

Reader. The default inbox (and the mailbox unread badges) exclude spam-verdict rows; a Spam entry in the per-mailbox folder rail shows only them. Per conversation, Report spam (inbox) / Not spam (Spam view) set the verdict directly.

Content scanner (rspamd)

The content layer is a second verdict source OR'd into classifySpam(): a message is spam if the content scanner flagged it or the auth rule fires. It changes no downstream behavior — same iem_spam_verdict, same Spam view, same forward suppression. The signal is resolved per ingest path:

• Postfix path. rspamd runs as a Postfix milter after opendkim + opendmarc (so it scores on the auth results), in header-stamping mode only (never rejects — consistent with the reviewable-verdict model). It stamps X-Spam: Yes on a spam verdict (plus X-Spam-Status carrying the score); InboundEmailRouter::readSpamHeader() reads that header, trusting it on the same basis as the Authentication-Results line (the milter is ours, and rspamd strips any inbound-forged X-Spam before re-stamping).
• Webhook providers. Mailgun, SendGrid and SES supply their own content/reputation spam signal in the authenticated payload; each provider's handleInbound() surfaces it as a spam key, carried into the router as a sibling of the auth verdicts. SES's spamVerdict and Mailgun's X-Mailgun-Sflag are binary verdicts. SendGrid posts only a numeric spam_score (its SpamAssassin score, no yes/no), so the binary is derived by comparing it to sendgrid_inbound_spam_threshold (default 5.0, SpamAssassin's own required_score; tunable on the Setup tab). The raw score is recorded either way.
• IMAP-polled mail. Unchanged — the remote already classified it (junk-folder mapping).

Learning. mailbox_spam_learning_enabled (default on, shown on the Settings tab as Learn from what users mark as spam, and only while filing is on) is the one advanced choice. It is the single capability no upstream scanner can provide: a Bayes corpus of this deployment's own mail, taught by its own users' corrections. A shared relay is deliberately stateless — one model trained across every tenant's mail would be both a privacy leak and a poisoning vector — so learning cannot be delegated upstream; it runs on the scanner that ships with the mail stack, whatever the topology.

Which mail is re-scored here. Relay- and webhook-sourced messages are scanned again locally at ingest through the controller's /checkv2, on any box where filing is on and a scanner is running. Learning is not a condition. An upstream scanner is stateless and its header is the only content signal a fronted deployment would otherwise ever get — and a header that was never stamped is indistinguishable from a clean verdict, so scanning here is what makes the difference observable. Colocated mail is not re-scored: its own milter already ran exactly that scan.

How much the local verdict counts is what learning changes:

How the scanner is used is decided in software by MailboxSpamPolicy, and every consumer (the health probe, the learning task, the ingest scan, the admin pages) asks it rather than re-reading settings or re-deriving topology. The provider is read resolved (InboundProviderRegistry::active()), never as the raw mailbox_provider row, so an empty or misspelled setting cannot flip an answer. learningEnabled() is clamped by filingEnabled(), so "learning with nothing filing" is unreachable rather than merely discouraged — the stored row survives as a remembered preference and takes effect again when filing returns.

Day 2: turning the scanner on and off. Installed-ness is observed, never declared, so the content_spam_scanner provisioner (InboundEmailHealth::checkContentSpamScanner) compares two facts — expected (localScannerExpected()) and present (the controller answers):

Spam/ham feedback (Bayes training). A reader correction (Report spam / Not spam) is the whole trigger — there is no separate "report" control. Flipping iem_spam_verdict leaves the row diverged from iem_learned_verdict (the marker of what was last taught). The LearnSpamFeedback scheduled task (every cron pass, gated on MailboxSpamPolicy::learningEnabled()) reconciles the divergence out-of-band: for each diverged row it POSTs the raw RFC822 to the controller's /learnspam | /learnham over loopback and, on success, stamps iem_learned_verdict = iem_spam_verdict so the row stops re-selecting. Flip-backs and idempotency fall out for free. Every correction that still has a raw message teaches the corpus, whatever path the message arrived by — webhook- and IMAP-sourced rows included, since the corpus is a deployment-wide asset and the local scanner is what scores that mail. Rows whose raw is gone (pruned, IMAP reference-backed, or sealed out of reach of this keyless cron pass) are marked handled as permanent no-ops. A controller that is unreachable — not yet installed, or down — returns skipped and leaves rows diverged to retry on the next pass, so the loop self-heals through an outage and rebuilds the corpus after a wipe rather than stranding corrections. (rspamd's classifier needs roughly 200 messages of each class before it contributes, so early corrections have little visible effect.)

Deliverability reports

Mail providers send machine-generated reports about a domain's mail — DMARC aggregate XML (who sent as the domain and whether it aligned), TLS-RPT JSON (where TLS to the domain failed), and ARF feedback-loop complaints (a recipient marked the domain's mail as spam). They arrive as ordinary email because the domain's published policy asked for them. DeliverabilityReportIngest (specs/deliverability_report_ingest.md) detects them during ingest and files them instead of delivering them.

Detection is by content, never by address. Two of three signals must match — the RFC 7489 attachment filename shape, the Report Domain: … Submitter: … subject shape, and the payload structure itself — so a misaddressed report is still caught and an ordinary message carrying a zip is never touched. ARF is recognised by its multipart/report; report-type=feedback-report content type. Detection runs at every moment the pipeline holds plaintext: receive time (InboundEmailRouter::processEmail, before the alias branch, and the relay pull path in RelaySpoolConsumer), and deferred parse at unlock (parsePendingMessage) for Fortress relay mail — sealed domains get the same inventory as everyone else because extraction happens while content is in hand, never against stored sealed rows.

A recognised report is filed, not delivered. No mailbox message is created (or, on the deferred path, the pending row is removed), so message counts, unread badges and quotas keep describing human mail. What persists is derived data: one dvr_deliverability_reports row per report and one dvs_deliverability_report_sources row per source line — IP, count, disposition, SPF/DKIM alignment verdicts, identity domains. Source rows never expire; they are the domain's long-term answer to "who has sent as us?". A parsed report's raw is discarded (the rows carry everything it said); a report that fails to parse keeps its raw in the report row for diagnosis, and a kind with no parser is recorded and counted rather than dropped. Every filing is a report_filed row in the transaction log.

Report content is untrusted input. Payloads are size-capped before and during decompression (zip and gzip both stream against a ceiling), XML with a DOCTYPE is refused outright and external entities are never resolved, and a report naming a domain this platform does not host is discarded without granting it anything. A failure anywhere files or falls back to ordinary delivery — it never aborts ingest of the carrying message.

Surfaces. The reports view (admin_mailbox_reports) is the sender inventory — sources over a chosen window, unaligned senders first, plus the report list with parse status. It has no tab of its own (the same reasoning as the relay: a diagnostic, not a daily workspace) — it is reached from the Setup tab's "Deliverability reports" row, which sits beside the DMARC row and shows reports actually arriving (the only proof the published rua address is right), from the new-sender notification email, and via the report_filed lines on the Logs tab. The first time a report names an unaligned source never seen for the domain, the domain owner gets one email — the source is then known and updates the inventory silently, with one escalation notice if its volume later jumps sharply.

Trash and retention

Deleting mail from the reader is a soft delete: MailboxService::softDelete() stamps iem_delete_time and nothing else. Trash is a view over that column, not a label or a folder — the same shape as the Spam view — so nothing has to move and a restore has nothing to reassemble.

Exactly one view sees a trashed row. Every read scope pins `iem_delete_time IS NULL; MailboxService::trashScopeSql()` inverts that pin, branch for branch (a single mailbox, an all-access "All mail", a superadmin per-domain "Unmatched" box). The Trash view is also the one view that ignores the spam verdict, so mail a filter trashed on arrival is not invisible in both places. listThreads() takes a trash filter, getThread() / messageIdsInThread() a $trashed flag; the reader passes them from its Trash rail entry.

Exactly two mutations reach a trashed row. restoreFromTrash() clears the column; purgeFromTrash() deletes for good. Both resolve targets through trashMutationScopeSql() and are its only callers — the IS NULL pin that every other mutation carries is what keeps discarded mail out of the read/star/archive/spam/label paths, so it is not a parameter those methods can be handed. Restore needs no bookkeeping: trashing never touched read, star, archive, spam verdict or label membership, so the message returns exactly as it left.

A purge reclaims everything or it is not a purge. Both the reader's Delete forever and the scheduled task go row by row through InboundEmailMessage::permanent_delete(), which frees the file-backed attachment fil_ Files and the stored raw object (local file or cloud object). A bulk DELETE would drop the row in one statement and leak both. Each id is queued for refold first, so the owner's sealed search index drops the entry at their next fold. Sealed mailboxes purge locked: permanent_delete() works on columns and storage keys, never on plaintext, so a Fortress mailbox needs no unlock window.

The window. InboundEmailMessage::purgeExpiredTrash() is declared in that class's $retention_policy and runs in the platform's daily retention sweep, purging what was trashed longer than mailbox_trash_retention_days (default 30) ago. 0 means nothing purges. A per-run cap (500) keeps a large backlog draining over several runs rather than one enormous transaction, and says so in its result message. Each Trash row shows when it purges, computed for display from the same setting and the row's delete time — never stored, because an operator can change the window.

Unmatched mail ages out on its own window. Mail stored for an address no alias claims sits in no member's mailbox, so nobody ever trashes it and the window above never sees it. InboundEmailMessage::purgeExpiredUnmatched() is the second rule in the same $retention_policy list, deleting stored unmatched mail received longer than mailbox_unmatched_retention_days (default 90) ago; 0 keeps it indefinitely. It reads its own setting because the two questions are different: a member decided about their Trash, and nobody ever decided about mail addressed to nobody. Read state is deliberately not consulted — unmatched mail is unread by definition, so exempting unread rows would exempt the category. Three kinds of NULL-alias row are excluded: already-trashed ones (the Trash window owns those, on its own clock), outbound rows (a compose belongs to a member, not a mailbox), and pending-parse rows (still on their way to an alias at the owner's next unlock). Reclamation and the per-run cap are shared with the Trash purge.

IMAP-backed mailboxes are one-way. ImapSyncer::pushTrash() moves the source copy into the account's Trash folder and repoints the locator (which doubles as the already-trashed marker). Restore and purge act locally: a restored message returns here while the source copy stays in the provider's Trash, and a purge deletes this row and these bytes without expunging anything remote. Providers run their own 30-day Trash purge, so the remote copy goes on its own schedule; the reader's Trash view says so on a mailbox that has a feed.

AI security scan

A danger score (0-10) plus specific red flags and a one-line summary, generated by the email_security_scan pipeline job (see plugins/joinery_ai/docs/overview.md § Registered jobs) for mail that passes the spam/auth filters above but is malicious in content — attacker-triggered notifications sent through a legitimate provider's own infrastructure (dmarc=pass) whose payload is an open-redirect sign-in link, which authentication-based filtering structurally cannot catch. The scan runs after delivery and only annotates; nothing is deleted, moved, or forwarded by it.

EmailSecurityDigest (includes/EmailSecurityDigest.php) is the deterministic, LLM-free reduction of one stored message to the bounded evidence the job's checklist prompt needs — a fixed-section plain-text digest (=== EMAIL DIGEST === header block, decoded FROM/REPLY-TO/RETURN-PATH/ TO/DATE, the stored AUTHENTICATION verdicts plus the DKIM signing domain, every extracted URLS FOUND — each with its visible anchor text when it differs from the href, so the classic link-text/destination mismatch survives tag-stripping, preceded by a DOMAINS: per-host count summary (top 15, most frequent first) so a small model never has to aggregate the raw URL list itself — and the decoded BODY). Headers not already stored as columns (Reply-To, Return-Path, Date) are read from the raw message when available; From/To/Subject fall back to the already-decoded iem_sender/iem_recipient/iem_subject columns when raw is unavailable (a remote-driver IMAP row). Whitespace and invisible-character runs of 4+ collapse to one space, with the removed count annotated once it exceeds 200 — turning obfuscation itself into citable evidence instead of context filler that can blow a small model's attention. Subject and body are each size-capped (1024 / 4096 characters) with a [truncated, N characters total] marker; URLs are capped at 20 with a (+N more) marker. EmailSecurityDigest::build() is a pure function of an InboundEmailMessage — no LLM concepts in the class. Its format is corpus-validated for this job (any change requires a full re-score against the labelled corpus), so email_security_scan reads it alone, unaugmented, until that re-score happens.

EmailAttachmentDigest (includes/EmailAttachmentDigest.php) is a sibling builder the email_triage and email_schedule jobs append after EmailSecurityDigest::build() — an ATTACHMENTS (N): section listing every non-inline attachment (up to 10, then a (+N more attachments) marker): a metadata line always (filename — content-type, size bytes, filename whitespace-collapsed and capped at 120 characters), plus, for file-backed parts only, readable text — a text/plain body (collapsed, capped at 2000 characters per part) or a text/calendar/.ics invite parsed with IcsImporter::parse() and rendered as a deterministic ICS EVENT: block (title, start with its timezone, end, location, organizer). All attachment text combined is capped at 4000 characters, with the same [truncated, N characters total] marker style EmailSecurityDigest uses. A section-pointer or IMAP (remote) part gets its metadata line only — no on-demand IMAP fetch from an unattended job. Any read failure degrades to [content unreadable] and a malformed .ics to [calendar attachment could not be parsed], each after the metadata line, never failing the item. EmailSecurityDigest stays untouched and corpus-frozen; opting the scan job into attachment evidence is possible but only alongside a corpus re-score.

Verdict fields, written only by the job's recordVerdict() (not $ai_writable_fields — there is no other write door):

• iem_ai_danger_score (int2, 0-10)
• iem_ai_scan (jsonb: {verdict, red_flags, summary, model, recipe_id})
• iem_ai_scan_time (timestamp)

Rules that match incoming mail and apply actions to it automatically — the inbound-email equivalent of Gmail's Filters and Blocked Addresses. Managed one mailbox at a time: a mailbox picker scopes the list, and Create filter is pre-scoped to the picked mailbox. The picker lists only mailboxes where filters can actually fire — those that store locally-received mail (delivery mode store / forward-and-store, and not IMAP-backed); IMAP-polled and pure-forward mailboxes are omitted because the filter hook never runs for them.

Two mounts, one code path. The Filters admin tab (between Accounts and Logs) is the operator surface: every mailbox on the deployment, plus each domain's All mailboxes in <domain> bucket for domain-wide rules. The member page at /profile/mailbox/filters, reached from the gear menu on the mailbox and from the Email settings section, offers the mailboxes that member holds grants for and no domain buckets — a rule that acts on every mailbox in a domain is an operator's to write. A filter can only be opened, toggled, deleted or saved from a mount whose picker offers its scope, so an id typed into the member URL reaches nothing of anybody else's. Both mounts render includes/mailbox_filters_panel.php from logic/mailbox_filters_logic.php; the mount passes its own base URL and whether it is the operator surface.

Scope. Filters run on freshly-received mail only — the Postfix milter path and the provider-webhook path, both of which funnel through InboundEmailRouter::storeMessage(). Because storeMessage is the single such path, the ingest hook there covers Postfix and webhook identically with no per-path branch.

Two ingest paths are exempt, for the same underlying reason: the mail did not just arrive, so acting on it would fire forwards and notifications for messages nobody received today.

• IMAP-polled feeds mirror an upstream account that already applies its own filters, and the reader's two-way sync treats the remote as the source of truth for flag/label state. They use storeExtracted(), which has no filter hook at all, so the exemption is structural.
• Archive imports pass run_filters => false to storeMessage(). An archive already reflects whatever filtering its source applied, and a decade of mail run through live rules would act on all of it at once.

• Criteria — From, To, Subject, Has the words, Doesn't have, Size (greater/less than a value + unit), and Has attachment. From/To accept comma-separated terms (any one matches); Has the words requires every word, and Doesn't have excludes any. A filter matches when all non-empty criteria match. At least one criterion is required.
• Actions — apply a label, star, mark read, Skip the Inbox (archive), mark as spam, never send to spam, forward to an address, delete.

Engine. The match and action logic lives on the InboundEmailFilter model. runForMessage() loads every in-scope enabled filter (the mailbox's own plus the domain-wide ones), runs matches(), accumulates the actions of all that match, and applies them once in a fixed order so multi-filter interactions are well-defined: never-spam → mark-spam → label/star/read/archive → forward → delete. An explicit never send to spam always beats mark as spam. It runs at ingest after the spam verdict is set, so a filter is the last word on disposition; it writes the state columns and label memberships directly (system authority), reusing the same primitives the reader uses. A forward action relays a copy through the same path alias-forwarding uses; a delete soft-deletes the stored copy last, so a forwarded copy still went out.

Archive ("Skip the Inbox"). The reader's default mailbox view is the Inbox (non-archived, non-spam, non-deleted); an All Mail rail entry shows everything, archived included. The open-thread toolbar offers Archive in the Inbox and Move to Inbox in All Mail — the manual counterpart to the filter's archive action.

Sent. A Sent rail entry lists every conversation carrying an outbound row — like Spam and Trash it reads a column (iem_direction), not folder membership, so it works for local and IMAP mailboxes alike. Row-level filter, thread-level effect: the thread's list row is its latest sent message and opening it shows the full history. Trash still wins (a discarded sent message is in Trash and nowhere else), a search in Sent is bounded to sent mail (an explicit scope), and spam reporting is not offered there — a verdict on the member's own outbound mail means nothing.

Timestamps coarsen with age. Both places the reader prints a message time — the list row and the open message header — run the same ladder: just now and N minutes ago under an hour, 3:45 pm under twelve hours, 3pm Jan 3 under six months, Jan 3, 2020 beyond. The hour and meridiem are composed rather than delegated to toLocaleTimeString, which would render 24-hour under some locales and disagree with the rung below it; the month name stays locale-aware (specs/mailbox_timestamp_ladder.md).

Sent and Drafts are ordered by time alone. Every other list is sectioned — unread first, then starred, then the rest — which answers what still needs me? On mail the member sent or wrote there is no such question: an outbound row's unread flag is whatever the source's \Seen said when it was pulled, or the ingest default of false, and never something the member decided. So those two views drop the sectioning and read strictly newest-first, the way every mail client shows sent mail. The mailbox unread badge excludes outbound rows for the same reason, and because the Inbox it lands on has never listed them (specs/bugfix_sent_view_ordering.md).

Apply to existing. A filter saved with Also apply to matching existing mail sets a pending flag drained by the ApplyInboundEmailFilters scheduled task, which pages through that mailbox's locally-received, non-deleted history in bounded batches and applies the same matcher and actions (forwarding is never re-applied to historical mail), resuming across runs via a per-filter cursor.

Logging. Each ingest that matches at least one filter writes a filtered line to the inbound transaction log (the Logs tab) recording the matched filter ids and the actions taken.

Importing from Gmail. The Filters list has an Import filters button that ingests Gmail's mailFilters.xml export (Gmail → Settings → Filters and Blocked Addresses → Export) into the picked mailbox. The file is uploaded and answered with a preview — one row per Gmail filter with its synthesized name, mapped criteria, mapped actions, and a Skipped column for anything that has no platform equivalent — then confirms to create the checked rows. An imported filter is an ordinary InboundEmailFilter; import adds no new behavior.

• Criteria map directly: from, to, subject, hasTheWord → Has the words, doesNotHaveTheWord → Doesn't have, hasAttachment, and size.
• Actions map directly too: archive, mark-read, star, trash → delete, never-spam, and forward. Gmail's label action find-or-creates a custom label by name (a nested Parent/Child name is kept verbatim); new labels are created on confirm and their count is shown in the summary.
• Skipped: importance (shouldAlwaysMarkAsImportant / shouldNeverMarkAsImportant), categories (smartLabelToApply), and chat exclusion have no platform concept and are dropped visibly, listed per row in the preview.
• The size default caveat: Gmail emits a default sizeOperator/sizeUnit on every exported filter even when no size is set, so a size criterion is imported only when a size value is actually present — otherwise every filter would gain a bogus "size < 0 MB" rule.

includes/email_providers/MailgunProvider.php
    implements EmailServiceProvider, InboundEmailProvider

includes/email_providers/SmtpProvider.php
    implements EmailServiceProvider       (outbound only)

includes/email_providers/PostfixProvider.php
    implements InboundEmailProvider       (inbound only)

A polled mailbox is modeled as a normal alias@domain: the address you poll is the mailbox. So a Gmail you read becomes the domain gmail.com with the ied_is_imap_source flag set (Setup skips MX/DNS for it) and [email protected] as a mailbox under it, fed by an IMAP feed. Multiple polled Gmail accounts sit as sibling mailboxes under the one gmail.com domain.

Per-host matrix — who needs OAuth vs. an app password

iia_last_status and the scheduled task's last-run message are both overwritten every pass, and a full-history backfill is hundreds of passes — so neither can answer what did the import lose two hours ago. Every poll that did something therefore leaves a durable row in evl_event_logs under the event mailbox_imap_ingest, holding the counts (seen, stored, duplicates, out of scope — the day-window guard's bucket — source drafts — the bucket for a message the source still holds as a draft — and failed) and each distinct failure reason with the number of messages it hit. Fifty messages failing the same way read as one line, not fifty.

Two things make an otherwise-silent loss visible:

• unaccounted — seen counts every UID the window walked. If stored + duplicates + out of scope + source drafts + failed does not reconcile against it, the shortfall is named in the note and the row is marked unsuccessful. This is the only signal for a message that disappeared without anything reporting a reason. A scope-guard skip and a source-draft skip are first-class buckets for exactly this reason: provable-on-purpose, never unaccounted.
• A UID the server returned no data for is a counted failure rather than a skip, so the reconciliation above stays honest.

A message and its attachment manifest are one write. Both go in a single transaction, so a poll interrupted part-way through leaves nothing behind rather than a message that lists no attachments. This matters beyond tidiness: the cursor has not advanced past an uncommitted message, so the retry stores it whole, and every other path can treat "the message is here" as "its attachments are here too". A collision the database raises — as opposed to one caught by the pre-validate check before any insert — surfaces as InboundStoreCollisionException rather than a reported duplicate, because Postgres has aborted the transaction and the caller cannot carry on inside it. The message rolls back, the UID retries, and the retry resolves the same collision cleanly.

Proving where a day-windowed feed started reading

A feed set to Last N days seeks the oldest message still inside the window and starts just below it. That decision is made once per folder and it decides what the user will and will not receive, so each seek writes a row to isp_inbound_imap_seed_proofs: the cutoff and high UID that went in, the cursor that came out, which method answered (isp_method — search for a server-side UID SEARCH SINCE, bisect for the band-probe bisection), how many probes it took, whether it converged or ran out of budget, and two boundary probes.

• below — the newest message at or under the cursor. Its date should be older than the cutoff. If it is not, the seek started too high and skipped mail, and the row is written to the error log as well.
• above — the oldest message over the cursor, which should be inside the window. This measures tightness, not safety: over-importing is the fail-soft direction and costs only time.

php maintenance_scripts/dev_tools/imap_window_audit.php --account=12

Every message view shows a clickable attachment list (filename, size, type), built from the ima_inbound_message_attachments manifest. For IMAP (remote) mail the bytes stay on the server — clicking one fetches exactly that MIME part on demand (FETCH BODY[<section>], Message-ID fallback if UIDVALIDITY changed), decodes it, and streams it pass-through with `Content-Disposition: attachment + X-Content-Type-Options: nosniff`. For push (Postfix/Mailgun) mail the part is a private File streamed the same way. Inline (cid:) parts belong to the HTML body and are excluded from the list. If a part can't be retrieved (message deleted/moved/account disabled), the endpoint says so honestly. The manifest + endpoint + reader list are transport-agnostic: the download dispatches on where the bytes live — a File for push mail, an IMAP fetch for remote mail, a raw section for a legacy/fallback row (see Attachment & message storage) — through the same endpoint, same table, same UI. The whole-message .eml is a separate export, offered only where a stored original exists — see The message kebab.

Deleting a feed with mirrored mail. Because a remote message's attachments live on the source account, removing the IMAP feed — or permanently deleting the mailbox that owns it — presents a keep/remove choice rather than silently stranding those rows (admin_mailbox_imap_delete). Keep materializes each mirrored message into a self-contained local copy (fetch the full RFC822 while the account is still connected, split attachments into private Files, drop the IMAP locator) via InboundEmailRouter::materializeRemoteMessage, then removes the feed; it requires the account be connectable, and refuses the delete if any message can't be copied so nothing is lost. Remove permanent-deletes the mirrored rows (the mail stays on the source server) and removes the feed. A feed with no reference-backed mail deletes directly, no prompt.

Setting up a Gmail account (end to end)

The live connect/fetch path is wrapped behind Horde; unit tests cover the platform side (model + encryption, reference-backed store + dedup, manifest + grant parity, poller summary). To connect a real Gmail account:

1. Gmail prep. In Gmail: Settings → Forwarding and POP/IMAP → Enable IMAP → Save.
2. App password. With 2-Step Verification on the Google account, create an app password at myaccount.google.com/apppasswords — the wizard's How do I get this? modal links every step.
3. Connect. + Connect a mailbox on the Accounts tab → Gmail / Google Workspace → enter the address and the app password, choose the reader and the protection level → Connect → finish the configure step (folder, history, name).
4. Verify. Click Test, then Fetch now. The first fetch seeds the cursor to "now" and ingests nothing — send a new email to the Gmail afterward, Fetch now again, and confirm it appears under the mailbox in the Mailboxes reader; open it and download an attachment. For hands-off fetching, activate the Fetch inbound IMAP mail scheduled task.

Each IMAP feed has a Sync mode, set per feed on the mailbox editor and off by default:

• Off — one-time import. The source is never written to and local read/star/ delete state stays in Joinery.
• Read-only — Joinery follows the source: a read/star/move/delete made in the native client is reflected in Joinery; Joinery never writes back.
• Two-way — full reconciliation: acting in either place is reflected in the other.

Custom labels are rows; folders are bindings. A custom label is an ilb_inbound_email_labels row, and a message has it iff it carries an ilm_inbound_label_members row with ilm_present_local — the same truth for locally-received and IMAP mail. An IMAP folder (iif_inbound_imap_folders) is a binding that mirrors one label to a remote folder on one feed (iif_ilb_inbound_email_label_id). Special-use folders (Inbox, Sent, Trash, Junk) and the \All coverage view bind no label — their state is a message column, not a label. The membership row is also the IMAP shadow: ilm_present_base records whether the message was in the bound folder at the last sync, alongside the folder UID, so truth and shadow share one row. Adding a label is a COPY (a Gmail label add) on a multi-folder host or a MOVE on a classic one-folder host; removing is `STORE \Deleted + EXPUNGE; deleting is a MOVE/COPY` to Trash. Operators pick which folders are tracked on the mailbox editor; special-use folders are pre-selected.

Changing labels from the reader. The open-thread toolbar has a Move to (folder icon, exclusive feeds) / Labels (tag icon, non-exclusive feeds, e.g. Gmail) control: pick a folder to relocate the thread, or toggle label checkboxes. Each change applies or removes the custom-label membership (MailboxService::setMembership, via the set_membership action); when the label is bound to a Two-way feed the next sync pushes the change to the source, and an unbound (local) label is pure membership that never touches a remote.

The list's selection toolbar carries the same control for every ticked conversation. Each list row arrives with label_ids (the custom labels any message in the thread carries — the same union the thread endpoint's folders reports), and a label's box reads the selection from them the way Gmail's does: ticked when every selected conversation has the label, mixed (indeterminate) when some do, clear when none does. Ticking puts the whole selection in, clearing takes it all out; the rows in hand are updated and the panel stays open, so several labels can be set in one visit. The one list change a label edit makes is taking the open label off the selection, which removes those rows from that label's view.

Creating a label/folder. The same control has a New label… / New folder… field, and is present on every mailbox — one with no labels yet shows the button with just that field, since it is the one place a first label is made. Creating one makes an ilb_ label; on a mailbox with an IMAP feed it also makes a tracked binding flagged iif_pending_remote_create (the remote folder does not exist yet) and files the thread into it. The folder is materialized on the source during the sync push — ImapSyncer issues the IMAP CREATE, clears the pending flag, then COPYs the message in; pull/ingest skip a pending folder until it exists. Creation is idempotent (a folder that already exists is adopted). Conversely, a label created on the source is discovered each sync as an untracked folder — tick it on the mailbox editor to start syncing it.

Deleting a label. Each row of the Labels panel carries a trash can at its right edge, shown when the row is hovered (always shown on a touch screen). It opens a confirmation that states the consequences and takes the label's name typed back before the Delete label button enables; the act is not reversible. delete_label (MailboxService::deleteLabel) removes the label from the site: a label is one shared name across every mailbox, so it comes off every message that carries it (the ilm_ rows are dropped) and out of every mailbox's list, and the ilb_ row is soft-deleted. No message is deleted. Every feed folder bound to the label is unbound and untracked, so the next sync neither re-materializes memberships nor re-mints the label from the folder name (rediscovery never flips iif_is_tracked on an existing row); the folder/label on the source itself is left as it is — nothing is deleted on a provider. A list filtered to the deleted label falls back to the mailbox's Inbox. A filter whose action applies the label keeps matching and simply skips the label. The name is free again: the uniqueness index on ilb_name covers live rows only, so creating the same name afterwards makes a fresh label.

The \All coverage view (Gmail All Mail). An all-mail folder is tracked as a coverage source, not a navigable label: it ingests every message — including mail archived with no label — so nothing is missed, but it carries no label. It is also where the source's own drafts surface, which is why the \Draft skip above is folder-agnostic rather than a matter of leaving the Drafts folder untracked. In the reader, the mailbox root is the label-unfiltered “All Mail” view, so messages with no label are reachable there; the labels listed beneath it narrow to one label.

Reconciliation. Each cycle runs Pull → Ingest → Push on one connection. Flags are a three-way merge keyed on a per-row dirty signal (a local change since the last push wins over an incoming remote change). Custom-label membership is reconciled through the single ilm_ row: an element is dirty when ilm_present_local differs from ilm_present_base — a column predicate a partial index covers, so the push scans only the dirty rows. Each (message, label) bit is a conflict-free boolean merge; on a one-folder host a divergent move converges to the local destination within two cycles with no explicit tiebreak. A pushed change is re-read next cycle as a value-equal no-op, so nothing loops.

Deletion (a separate “Also sync deletions” toggle): driven by the iem_delete_time column, not a label. A local delete moves/copies the source message to Trash (the locator follows, so it is never re-pushed); a message arriving in Trash on the source soft-deletes the local row at ingest. Archiving (the iem_is_archived column) is distinct from deletion and stays local.

Compose / Sent (the “Enable compose / Sent sync” toggle, on by default in the connect wizard whenever sync is on, with the reader’s reply/forward feature): the source Sent folder is ingested like any tracked folder, so mail sent from the native client appears in Joinery. When a feed’s SMTP does not auto-file sent mail (self-hosted / generic), Joinery APPENDs the sent copy to the source Sent folder itself. Sent dedup is by Message-ID only: a compose send always stores its own outbound row, carrying the Message-ID the message was sent with, and the filed copy reconciles to that row on the next Sent ingest — adopting its IMAP locator rather than inserting a second row.

A message filed in the source Sent folder reads as sent mail (`iem_direction = 'outbound') whichever folder stored it first: when the \All` coverage pass wins the race and stores it as an ordinary row, the Sent pass's dedup promotes that same row. A self-addressed message is the exception — it stays inbound and shows in the Inbox, exactly as the source mailbox files it. (A self-send this deployment composed has no such row to promote: it reconciled onto the composer's copy at ingest.) The promotion only ever lifts an inbound row (outbound and draft rows are never touched) and stands down when a live outbound row already holds the same (Message-ID, recipient) dedup key.

Provisioning a mailbox in one call

plugins/mailbox/includes/provisioning.php exposes the headless provisioning functions — register a domain, create a store-mode mailbox on it, and grant it to a user, with no page or session plumbing:

• mailbox_provision_domain($domain_name) — ensures the domain row exists (files fleet domain claims on creation, exactly like the Setup tab's add-domain action).
• mailbox_provision_mailbox($domain_name, $local_part, $user_id) — domain + store alias + access grant in one call.

{"ok":true,"dkim_ready":true,"records":[{"type","name","value","priority","note"}]}

php maintenance_scripts/dev_tools/takeout_parts_probe.php /path/to/export/

Provider conventions are read where present and cost nothing where absent: Proton's <id>.metadata.json sidecars, Gmail's X-Gmail-Labels header (where read state is the presence of Unread), and maildir's :2, filename flags. A bare folder of .eml files with none of these still imports correctly, just with less state.

Where a message lives, and what is merely laid over it

A message has one location and any number of labels, and an export gives both in the same list. Telling them apart is what decides the whole import: a view like Proton's All Mail sits on every message in the account, so reading one as a folder files an entire archive under a single heading and loses the real one.

MailArchiveReader::PROTON_LOCATIONS is therefore an allow-list of the ids that name a place — Inbox, Trash, Spam, Archive, Sent, Drafts, Outbox. Everything else numeric is a view (All Mail, Almost All Mail, All Sent, All Drafts, All Scheduled, Snoozed, the inbox category tabs) and is dropped. Dropping is deliberate on both counts: a view is not a place, and an unrecognised number used as a name is how an import ends up tagging thousands of messages with a label called 15. Where two locations somehow appear together, PROTON_LOCATION_PRIORITY decides — thrown away beats filed away, so Trash and Spam win and the message stays out of an import that did not ask for them.

The export's labels.json names every custom folder and label, and its Type field says which is which — 3 for a folder, anything else for a label. A custom folder is a location in its own right, so a message in one lands there rather than nowhere. A custom label is applied as a label and leaves the location alone. Without the manifest, custom entries are treated as labels: the name survives, the placement does not, and a bare folder of .eml files still imports.

Formats that carry no folder information of their own — a lone mbox, a single .eml — name the folder after the archive itself. That name is the one the person typed (mir_source_name), never the path on disk: the file store appends a uniquifier to keep names from colliding, and importing Receipts.mbox must not produce a folder called Receipts a7f3k2q1.

The two sources

Upload an archive, or pick a file already in your files. There is no server-path option — pointing at a folder on the machine is not something a member can do, and an uploaded archive becomes the user's own file the moment it lands, so an interrupted run resumes against it rather than needing a re-upload.

Uploading has no size limit. The archive goes up through the platform's resumable chunk transport under the mail_import_archive upload purpose (docs/api.md § Uploading something that is not a Drive file), so the bytes never ride in a single request and upload_max_filesize never applies. A dropped connection resumes from the server's byte count rather than starting the archive again, which on a large export is the difference between a hiccup and an afternoon.

A file in an encrypted folder is listed but refused, with the reason. Drive encryption is per-folder and inherited, and an encrypted file's plaintext exists only in the browser — the server genuinely cannot read it. It is shown rather than hidden because a user who cannot find their archive is worse off than one told why.

One at a time

A person may have one import going at a time. While they do, the start form is not on the page: in its place is a line naming the archive that holds the slot and what it is doing. The form returns by itself when that run finishes — the page is already polling, so no reload is needed.

A run counts as going in every state except done, failed and undone. That includes scanned, where nothing is moving because the run has stopped to ask which folders to bring: it resumes on the answer, so it holds the slot until it gets one.

The rule is scoped to runs the caller started, not to the mailbox. An operator setting up somebody else's mailbox is still the person doing the importing, and two grantees of one shared mailbox are two people.

MailImportService::activeRun() is the single answer to is this person busy. The page render reads it, mailbox/mail_import_status returns it as busy_run so the poller can flip the form back without re-deriving anything, and mailbox/mail_import_start refuses on it — so a second browser tab cannot queue what the first is already carrying.

Where the form picks up from

Importing is rarely one archive. A Takeout arrives split across several files and a provider migration means the same mailbox and the same address list over and over, so the form opens on the last run's answers: the mailbox that run targeted, and the addresses declared for it. An explicit ?alias_id still wins, a mailbox the caller no longer holds is not re-offered, and asking about a different mailbox falls back to the suggestion for that one rather than carrying across a list written for another.

Declaring your addresses

An archive carries no envelope. Without knowing which addresses were the user's, there is no way to tell sent mail from received, and no way to know which of several addresses a message actually reached. So the run asks — pre-filled from the last import when there was one, otherwise from the account — and derives two things from the answer:

• Direction — mail from one of those addresses is mail you sent. The source's own filing outranks the headers: a message sitting in Sent was sent, even if its From is an address the user forgot to declare.
• Delivery address — the first of Delivered-To, X-Original-To, Envelope-To, then To/Cc, that names a declared address. Nothing matching falls back to the target mailbox's own address, which is the honest answer for a Bcc.

On completion the run holds at scanned and the user picks folders, with Spam and Trash unticked: an archive's spam folder is usually the largest thing in it and almost never what anyone meant to keep. Anything left out is marked skipped rather than deleted, so the final reconciliation can still account for every message found.

Storing

Batches of entries, oldest first. Almost none of this is new code — live delivery already parses bodies, splits attachments into private Files, computes thread keys, seals content to the owner's vault, and treats a unique violation as a successful dedup — so the importer points the existing store path at a different source of bytes.

Two properties of the schema carry the design:

Dedup is free and correct. Re-running an import over the same archive stores nothing new, so resume-after-a-crash costs at most one batch, retry is safe, and "did I already do this" needs no bookkeeping. The importer asks whether this mailbox already holds this message id in this direction, which is stronger than the unique constraint alone: on a protected mailbox a sent message's recipient is sealed content and cannot be matched on a second pass.

Filing and delivery address are independent. iem_iea_inbound_email_alias_id decides which mailbox the message appears in; iem_recipient records where it was delivered. Mail can be gathered into one mailbox while each message still says honestly which address received it.

Messages with no Message-ID get a stable synthetic one, <sha256(raw)@import.invalid>, written into the stored copy so the row and its raw agree. It is derived from the bytes, so the same message scanned twice produces the same id and still dedups; .invalid (RFC 2606) can never collide with a real domain.

Imported mail carries its own Date header as its received time, so a decade of mail sorts where it belongs instead of landing all at once at the import's clock. A folder that is not one of the platform's own buckets becomes a label of the same name; the standard buckets (Inbox, Sent, Spam, Trash, Starred, Archived) are columns on the message and are handled by the store. Trash arrives soft-deleted, which is how the platform models a bin.

Imported mail carries no authentication verdict — unverified across the board. The stamps in an archived message were written by whichever server received it years ago, and this deployment cannot vouch for them. The reader says so in as many words ("Sender not checked — imported from a mail archive, so it never arrived here to be checked"), because on a deployment that has imported an archive this is the majority of stored mail and reads as alarming otherwise.

Attachments and where the bytes go

Every non-text part is split out into its own private File and linked from the message's attachment manifest, exactly as live delivery does — the importer inherits this by going through the same store path. Unlike an IMAP feed, which is reference-backed and fetches parts from the source on demand, an imported message is self-contained: the archive is the only copy of those bytes, so they are stored. A large Gmail archive is therefore a real disk commitment.

A message the mailbox already holds as references — ingested over IMAP — gets its attachment bytes too. The import still counts as a deduplication (no second copy of the message, no run tag, so undo still cannot remove mail the run did not create), but the bytes it was holding are kept rather than dropped. On a protected mailbox they land sealed, which works even though an import runs in the background with nobody signed in: sealing needs only the vault's public key. See Attachment & message storage for the matching rules and what is deliberately never upgraded.

Those Files are tagged email_attachment (fil_source), which is what keeps them out of the member's Drive listing and away from their Drive quota — an attachment is not something the user filed in their Drive, and a thirty-thousand-message import must not silently fill it.

The uploaded archive itself is tagged mail_import_archive: also not a Drive item, because it is working material for one run rather than a file the member is keeping. The file picker therefore offers Drive items and previously-uploaded archives, which is what lets an interrupted run restart against the same file instead of re-uploading gigabytes.

Undo reclaims all of it — attachment Files, manifest rows and any stored raw object go with the message, through the model's own permanent delete.

Any size

Both phases run in the RunMailImports scheduled task, because a 50GB scan cannot happen inside a web request. Each pass claims one run with an atomic conditional UPDATE — the same overlap guard PollImapAccounts uses — does one bounded batch, and returns. A claim goes stale after 30 minutes, which is how a run whose pass was killed gets picked up again rather than sitting claimed forever.

Scanning gets a time budget rather than a message count (it reads sequentially and writes narrow rows), and hands back an opaque cursor the reader understands: a byte offset for an mbox, a member index for a container. Storing gets mailbox_import_batch_size entries. mailbox_import_max_concurrent caps how many runs are underway deployment-wide so one enthusiastic user cannot starve the mail stack.

Progress is mir_processed against mir_total_entries, advanced by the importer with one atomic UPDATE per batch. Every write to a live run is a targeted column update rather than a model save, because the counters move underneath any model instance held for more than an instant.

Every entry is time-boxed. importBatch() arms a SIGALRM watchdog (MailArchiveImporter::ENTRY_TIMEOUT_SECONDS, 300s) around each entry, so a message that sends a parser into a loop becomes one failed entry with a reason rather than a cron worker pinned at 100% CPU holding the task's "already running" lock. The guard is CLI-only by nature (pcntl does not exist under FPM), which matches where batches run. Raw MIME parsing itself goes through core MimeParse::parseMessage(), which refuses the one input shape known to hang the Horde parser — a declared boundary quoted mid-line in the body — with MimeParseHazardException; the body walk then falls back to the legacy splitter and the message still imports.

The page shows when the next batch lands. The import panel's poll carries next_batch from MailImportService::nextBatch(): a countdown to the runner's next pass (from the cron runner's own measured tick spacing, scheduled_tasks_cron_observed_interval, plus its heartbeat) and how many messages that pass takes (mailbox_import_batch_size, capped by what remains). When the import task has been silent far longer than its cadence while a run is active, the page reports the worker as stalled instead of counting down to a batch that will not come.

The run record

Per-entry failures are recorded on the entry with a reason and never abort the run — one unreadable message must not cost the other thirty-five thousand.

Each batch that did something writes an evl_event_logs row under event mail_archive_import, with failures rolled up by reason so four hundred messages failing identically read as one line, plus bounded per-entry detail in the error log. This is the same MailRunRecord machinery the IMAP run record uses, with one extra bucket for what the user chose to leave out.

The reconciliation tripwire applies here too: stored + duplicates + skipped + failed must equal what was seen. A shortfall is reported as unaccounted and marks the batch unsuccessful, because a message that vanishes without a reason is exactly the failure a set of counters alone hides.

Every duplicate names what it duplicated. A duplicate is not one outcome, and the differences decide whether the message is actually in the mailbox:

python3 maintenance_scripts/dev_tools/mail_archive_inventory.py archive.mbox -o inventory.jsonl
php maintenance_scripts/dev_tools/reconcile_mail_import.php --run=42 --inventory=inventory.jsonl

• By Message-ID — source ids with no row in the target mailbox. Soft-deleted rows count as present, since mail the source had in its bin is correctly stored as a deleted row.
• By byte offset — a message with no Message-ID is stored under a synthesized id derived from its bytes, which the inventory cannot reproduce, so those are matched by position instead: the inventory records each message's body offset and mbox locators are offset:length against the same file. This also localises exactly where the two splitters diverged. Offsets are unique only within a file, so an archive holding several mboxes needs --member=NAME to say which one the inventory covers; without it the comparison is skipped and reported as skipped rather than guessed at.
• By attachment count — per message present on both sides, the source's part count against the stored manifest's rows. Catches a dropped attachment on a message that is otherwise fine. Count, not filename: the two sides name and de-duplicate parts differently.
• By ledger reason — the suspicious buckets above.

PurgeMailImportArchives (daily) collects archives whose run finished more than mailbox_import_archive_retention_days ago, and a Discard archive button on any finished run reclaims it immediately when the user knows they are done.

An archive picked from the user's own Drive is released, never deleted. It is their file, it counts against their Drive quota, and they may well want it afterwards — the importer only reclaims what the importer created, which is what the mail_import_archive origin tag is for.

The run itself always survives losing its archive: the record of what was imported outlives the file it came from.

Working directories (used only by formats that cannot be read in place — a zip holding an mbox, a tar) are removed when a run finishes. The same task sweeps any left by a run that ended some other way, which nothing else would collect.

Undo

Available on a finished run. It permanently deletes every message carrying that run's id — through the message model's own permanent delete, so attachment Files, manifest rows, label memberships and stored raw objects all go with it — and removes labels the import created that are now empty.

Mail that deduped against something already present was never tagged, so undo cannot remove mail the import did not create; neither can it touch anything that arrived afterwards. Labels that existed beforehand, or that still hold mail from elsewhere, are left alone: undo reverses the import, not the user's filing.

The run itself moves to undone and keeps its entries, so the report of what happened outlives the reversal.

Surfaces

Member — Import old mail at /profile/mailbox/import. Admin — the same tool in the Accounts tree beside IMAP feeds, at /plugins/mailbox/admin/admin_mailbox_import, with a mailbox picker covering every mailbox.

Both render the same panel and call the same logic; the only difference is which mailboxes are offered, decided by MailImportService. A member must hold a live grant on the target mailbox; permission 5+ may target any. That check lives in the service, not in a view, so calling the API directly cannot bypass it.

Actions are mailbox/mail_import_start, mail_import_status, mail_import_select and mail_import_undo on /api/v1, with the browser-session credential. The start action accepts multipart/form-data so an archive uploads in the same request that starts the run.

Settings

Adding a format

One class extending MailArchiveReader (or MailArchiveTreeReader, if the format is a tree of members) and one line in MailArchiveReaderRegistry::READERS. Order in that list is priority: readers are asked in turn and the first to claim a file wins, so the list runs from the most specific sniff to the loosest. A reader answers three questions and no others — is this file mine, what messages are in it, and give me the bytes at this position. The locator it hands out is private to it; nothing else ever interprets one.