User manual →
The current board, adaptive case briefings, service health, research, decisions, exports, and Handled memory.
Everything needed to install, operate, administer, and eventually tune Nightwatch—without crawling behind the rack.
Nightwatch has three explanatory rooms. After Hours explains the product, Behind the Rack explains the machinery and cost, and At the Chalkboard exposes the reasoning order. This field manual covers installation, current deployment, operation, extension, and safety.
The current board, adaptive case briefings, service health, research, decisions, exports, and Handled memory.
Deployment, identities, targets, secrets, models, notifications, retention, and safe writes.
What deterministic hunts and statistical signals do today, plus the intended rule-authoring boundary.
The contract for future bounded investigation methods—not controls that silently exist today.
How Nightwatch explains a conclusion and how wire evidence becomes a governed, growing case.
Nightwatch is a Python 3.10+ process with no pip dependencies. It needs at least one compatible evidence connector and a tool-calling model endpoint. The current distribution can use its wire-data adapter; the core is not tied to that source.
qrencode for locally generated authenticator QR codes; manual TOTP secret entry remains available without it.Packaged release (recommended for operators): obtain the platform archive and checksum, verify the checksum, unpack it, and use the bundled service template. A connector may be packaged separately or included by the distributor. Releases never include credentials, customer data, endpoints, or a working site configuration.
sha256sum -c SHA256SUMS
tar -xzf nightwatch-VERSION-PLATFORM.tar.gz
cd nightwatch-VERSION-PLATFORM
cp nightwatch.example.json nightwatch.json
# Add secret-file references and environment-specific settings.
python3 nightwatch.py --doctor --config nightwatch.json
python3 nightwatch.py --config nightwatch.json
Source installation: clone the repository, copy the credential-free example configuration, install a compatible connector, and run the read-only preflight before initializing the service.
gh auth login
git clone <nightwatch-repository>
cd nightwatch
cp nightwatch.example.json nightwatch.json
Open http://127.0.0.1:8321 after startup. Put a maintained TLS reverse proxy in front of Nightwatch before allowing remote access.
--doctor is read-only. It checks the executable and checksum, database, target authentication, record and metric coverage, lamp groups, telemetry freshness, model tool calling and pricing, bind/auth posture, secret-file modes, and disk capacity. Treat a failed required check as an installation failure.
nightwatch.json, nightwatch.db, and the evidence directory; deploy the new release as a unit; run --doctor; then restart. Never replace the database with an empty release artifact.The supported small deployment is one dedicated application host with local persistent SQLite and evidence storage, fronted by a maintained TLS reverse proxy. It is inexpensive, inspectable, and appropriate for a handful of concurrent analysts.
Do not begin with a cluster merely because one may be needed later. First move the production runtime away from the build host. Then increase CPU, memory, and disk. When real concurrency or retention requires it, separate web/API from background workers and move durable state to a transactional database and object storage behind the same repository interfaces. Case identity, audit, connector receipts, and workflow semantics must not change during that migration.
Nightwatch is one operating desk built from shared evidence. It does not require separate SOC and NOC roles. One person may investigate an account, restore a service, decide whether the two are connected, and verify the result.
| Shape of work | The question it answers | Typical action |
|---|---|---|
| Case | What condition deserves ownership, what supports it, and what happens next? | Investigate, restore, contain, confirm, dismiss, or keep watch. |
| Investigation | What question are we researching, and what has the evidence established? | Continue, pause, archive, link to a case, or propose a case. |
| Security campaign | Do several security findings form one connected threat story? | Preserve stages and provenance inside the case. |
| Service episode | What changed in a measured service during this bounded period? | Diagnose, restore, verify, keep measuring, or support a case. |
These are aspects of work, not organizational lanes, permissions, or personas. A familiar account doing something new can matter to a case; a machine where several symptoms converge can anchor an incident; a failing dependency can anchor a service. Nightwatch should name the thread and explain why it matters rather than present an inventory of accounts, hosts, addresses, and framework codes.
Overview is the morning briefing: what needs attention, what Nightwatch is doing, what it handled, and whether apparently separate symptoms may be connected. Ask Nightwatch can answer a quick wire-data question, start a durable investigation, accept a pasted brief, or independently test claims. A quick question does not become a case unless the evidence supports managed work.
A case card on Overview is deliberately a glance: urgency, state, one-sentence thesis, the thread that connects it when useful, compact scope, and the next action. A changed or connected marker may tell you why to reopen it. Evidence, framework identifiers, investigator history, and the full card spread belong in the Case Room.
Investigations is the file cabinet for work that is not necessarily a case: a threat hunt, service diagnosis, identity review, architectural question, pasted scenario, or claim challenge. Search active or archived work, see who started it and whether Nightwatch is working, waiting, paused, incomplete, or finished, then resume it without losing the transcript or evidence cutoff. Rename and archive keep work manageable; purge is a separate explicit destructive action.
An investigation can remain research, link to one or more cases, or propose a new case when its findings support a condition that deserves ownership or action. Linking does not copy the transcript or manufacture telemetry. Case-bound research and standalone research use the same durable workspace.
Every scheduled round, evidence-triggered follow-up, and human Run now or Update case now request is recorded before execution with a stable identity, target, reason, due time, prerequisite versions, priority, attempts, and fenced worker lease. Recurring work is anchored to UTC slots, so a slow or failed run does not shift the whole schedule. A human-triggered review wakes immediately but does not erase or postpone the normal cadence.
Collection wakes correlation; correlation and explicit case review wake Case Manager; the exact resulting case version wakes Daylight; a current Daylight result can then wake notification delivery. Failed work retains its error and deterministic retry time. Expired leases are recovered after restart, duplicate triggers are idempotent, and stale workers cannot publish. The activity display summarizes working, ready, retrying, blocked, and completed work without exposing internal tool chatter.
The Cases desk holds durable cases and the service evidence that may support them. Its All Cases, Needs You, On Watch, Active, Waiting, Resolved, Services, and Connected lenses are different views of the same operating desk. Needs You means a person owns the next decision. On Watch means Nightwatch owns the next bounded check. Active, Waiting, and Resolved make human progress and verification visible. Services can remain measured conditions without becoming cases; Connected shows meaningful case/service overlap without claiming that one caused the other.
A case appears as soon as the evidence supports a coherent proposition worth working. It can then gain findings, change confidence, acquire a different presentation, or receive another deterministic assessment. When exact evidence later proves that two visible cases are one story, Nightwatch collapses them into the older case, preserves which assessments joined, and explains why under Reasoning. It does not hide useful work while trying to build an ultimate campaign.
Nightwatch keeps two clocks. A one-off lead on Still on watch expires after a short complete clean window if it never develops. An established case stays active on a much longer case clock. Expiry and quiet resolution archive attention; they do not delete evidence. When new work shares exact evidence—or the same strong subject and activity family—an earlier deck can return as clearly dated historical context.
Selecting a case opens its first-class working room rather than stretching the Overview card into a report. Every entry point—Overview, Cases, a connected service, or a direct link—refers to the same durable case. The left side is the case folio: title, phase, owner, priority, confidence, lifecycle, current judgment, and the next owned move. The prominent action follows the phase: accept, record progress, resume, confirm resolution, or reopen. Nightwatch drafts routine progress from the existing case so the analyst can accept or edit instead of completing a ticket form.
Details and Investigator remain independent right-hand rollouts. Either can open beside the folio; when both are open they split the right workspace vertically, with horizontal and vertical resizing. Details holds proof and provenance. Investigator is the case-aware research surface. Neither creates a second case or a second source of truth.
The operational path is On Watch → Needs You → Active → Waiting → Resolved/verification → Handled, with Reopened when new evidence changes a closed judgment. Evidence state and workflow state are deliberately separate. Every transition records actor, time, previous and next phase, summary, owner, next action, and review or resolution context in append-only history.
Nightwatch presents an argument, not an evidence dump. The case composition adapts to the evidence: two sentences may be enough for a familiar recurrence; an ordered sequence may need a timeline; shared identity or device activity may need a relationship sketch; and a possible service connection may need aligned lanes. The card metaphor means “lay the useful facts on the table.” It is not a fixed deck, a game, or a requirement that every case have equal cards.
The investigation is navigable in layers: case → line of argument → contributing deck → source evidence. The case opens with Nightwatch’s plain-language conclusion, evidence spine, impact, uncertainty, and next move. “How this case came together” then exposes each narrower assessment as a deck and each observed finding beneath it as an evidence card. Nearby cases remain visibly outside the boundary until shared evidence earns a merge.
Every explanation follows the same disclosure order:
The first view contains only what is needed to understand and act. On desktop, the Case Room coordinates an adaptive canvas, an inspector for the selected card or relationship, and one persistent Case investigator. Nightwatch recreates the investigator from the current case file on every turn, carries a bounded handoff from older conversation, retains recent turns verbatim, and keeps the complete transcript for audit and export. The case can therefore grow for days without depending on an opaque provider thread or sending its entire history on every question.
Likely follow-ups that Nightwatch can already answer stay within reach—for example, “Why is this account unusual here?”, “Have we seen this before?”, or “Did the slowdown begin first?” Anything else belongs in the Case investigator. Opening stored evidence should not spend model tokens.
Nightwatch does not squeeze a spatial desktop canvas onto a phone. Mobile leads with case identity, thesis, and next action; then it presents one active visual or card and the suggested-next cards in a deliberate vertical order. Inspector and investigator controls open focused, accessible surfaces. Mobile Investigate means selecting, comparing, pinning, and asking—not dragging tiny cards around.
| Record | Use it for | Effect |
|---|---|---|
| Analyst note | Attributed local context, interpretation, correction, or a person-supplied fact. | May influence the next case composition, but Nightwatch must cite it as a human statement. It is not telemetry. |
| Investigator chat | Exploring a question with the case-owned investigator. | Changes nothing by itself. Explicitly save a useful answer as a note when it should inform the case. |
| Formal decision | Recording disposition or workflow action with a reason. | Changes case state and attention handling and retains author, time, and scope for audit. |
Nightwatch keeps human context separate from detections, records, and measurements. It may say “Aaron noted that this was approved maintenance”; it may not turn that note into “telemetry confirms approval.” If later evidence conflicts with a note, the case should show both rather than quietly choosing one.
| Decision | Use it when | Effect |
|---|---|---|
| Escalate / confirm | Evidence supports a real incident or needs response. | Records the analyst outcome and preserves the handoff. |
| Keep watching | The case is plausible but the discriminating evidence is not available yet. | Keeps the case On Watch with a bounded next check. |
| Dismiss / benign | The evidence supports an ordinary explanation. | Moves attention away while retaining audit history. |
Each lamp corresponds to configured source activity groups and a metric pack. The selected service leads with a plain-English conclusion, why it matters, Nightwatch’s leading explanation, what changed, and what happens next. A service episode is a bounded measurement period; it becomes part of a case only when the condition needs ownership, continued observation, or action. Measurements, named hosts and clients, counter-signals, resolved checks, packet evidence, and exports remain supporting material. History answers whether a condition is new, recurring, or worsening.
Service reasoning runs only on a material episode change. The deterministic measurement remains authoritative; one bounded read-tool turn may settle a precise unanswered wire question before Nightwatch publishes. It must not hand an available evidence-source lookup back to the operator as homework. Missing endpoint, change, ownership, or human context stays explicitly missing. Services do not create a parallel investigator or duplicate the Case Room.
When case evidence and service impact overlap, the Connected lens places the case and the measured service condition side by side and draws the exact shared subject and UTC window between them. A separate Nightwatch analysis explains how they connect, why it matters, and whether the relationship is direct wire confirmation, supported contribution, coincidence, or still unproven. Supporting evidence, counter-evidence, resolved checks, missing evidence, impact, and the deciding next step stay expandable. The case and service remain independently navigable; the view does not repeat one summary twice or silently turn overlap into cause. Full durable incident workflow remains in development.
Handled is durable attention control, not deletion. Repeated benign patterns are grouped with scope and history. A new host, account, or destination is judged again. Reopen a pattern when the business context changes; disable a suppression when it is too broad.
Administrators control connectivity, identity, cost, persistence, write permissions, and the definition of “inside.” Those choices change what every analyst sees.
The gear opens the authenticated control plane. People & access supports search, per-account changes, visible-row selection, and atomic bulk role or enablement updates. Disabling an account preserves authored decisions and audit history while revoking its active sessions. An administrator cannot remove their own access, and Nightwatch always retains at least one active administrator.
System configuration exposes eight validated categories: Environment & targets, Service lamps, Schedules & cadence, Investigation context, Notifications, Email & recovery, Authentication security, and Audience & capacity. Search locates a category; each editor saves the complete safe configuration while preserving fields outside its schema. It never resolves or returns secret values, writes a mode-600 backup, and replaces the configuration atomically. Restart Nightwatch after a saved configuration change.
Administrators can generate, replace, enable, and disable a hashed invitation code; the login page exposes account creation only while invitations are enabled. With SMTP configured, signup creates a pending account, sends a single-use 30-minute verification link, and activates a viewer only after verification. Password recovery uses an indistinguishable response for known and unknown addresses, a single-use 20-minute link, bounded request rates, and session revocation after reset.
Each user can review active sessions, revoke an individual sign-in, sign out everywhere, and enroll a standard TOTP authenticator. Nightwatch shows recovery codes once and stores only their hashes. Administrators can require MFA by role and configure session lifetime, secure-cookie behavior, login-attempt windows, lockout duration, and the accepted legal-policy version. The server-console password-reset command is the break-glass path: it re-enables the named local account, revokes its sessions, and clears MFA so the administrator can enroll again. External identity providers remain a future connector boundary.
Optional Cloudflare Turnstile can protect login, signup, and recovery. It is disabled by default and makes no Cloudflare request while disabled. When enabled, Nightwatch requires a public sitekey, secret-file or environment reference, approved hostnames, and selected forms; the backend verifies every token’s success, action, and hostname before authentication logic runs. Turnstile supplements rate limits, lockout, verified email, and MFA—it does not replace them.
Administrators can bound interactive requests per user and source network, manual actions per user, FIFO interactive concurrency, queue wait time, and deployment interactive cost per day and month. Quiet mode pauses analyst-triggered model work and forced rounds while deterministic scheduled measurement continues. Demo reset and viewer-export policy are disabled by default; when explicitly enabled, reset removes chats and saved notes while preserving cases, evidence, accounts, configuration, and audit history. These limits protect a shared lab and complement—not replace—upstream provider quotas and authorization roles.
Useful context comes from deterministic sources before another reasoning call. Nightwatch resolves exact typed public IPs through prefix, BGP origin, RIR registration, and RPKI, cached for seven days. An opt-in Spamhaus DROP source checks exact typed public IPs and ASNs against a 24-hour cached snapshot. A listing is supporting third-party context; no listing is not evidence of safety. Private, documentation, internal-marked, and prose-derived addresses never leave the system. ATT&CK, KEV, and EPSS context is also available. Observed DNS, configured resolvers, DHCP/IPAM, maintenance, CMDB, IdP, provider ranges, and additional governed sources remain next layers.
The EDR configuration is a provider-neutral stub. It performs no endpoint query, loads no endpoint credential, and enables no response write. Any endpoint adapter must satisfy the same evidence, health, privacy, and failure contract before Nightwatch calls it available.
Back up the configuration, SQLite database (using a SQLite-safe snapshot while running), evidence directory, and secret references. The database contains cursors, campaigns, feedback, outcomes, suppression memory, audit records, and session state. Test restore on an isolated bind before declaring recovery complete.
Record who changed a target scope, detector threshold, model route, retention window, or write capability; why; the expected effect; validation result; and rollback point. Run doctor after infrastructure or credential changes and after every release.
The example configuration is the canonical portable template. The table below groups controls by operational consequence; defaults are template defaults, not universal recommendations.
| Section | Controls | Operational meaning |
|---|---|---|
| llm.primary / fallback | provider, base_url, secret reference, model, provider options | Selects native Anthropic or OpenAI-compatible/OpenRouter routes. Use separate keys when budgets and failure domains must be independent. |
| llm.limits | daily requests, request spacing, fallback daily/monthly/per-call USD, cooldown, failure fallback | Cost and quota circuit breakers. They govern route eligibility, not detector collection. |
| llm | reasoning_queue_limit, tool rounds, temperature, retained tool results | Bounds concurrency, agent depth, sampling, and context size. Raise only after measuring quality and cost. |
| targets[] | name, host, auth, TLS, internal networks/domains, reconciliation scope, direct destinations | Defines evidence sources and identity boundaries. Wrong “inside” scope causes wrong attribution. |
| lamps[] | id, label, connector-declared groups, metric pack | Maps business-facing service areas to measurable activity. |
| context_sources | opt-in reputation provider; disabled EDR provider placeholder | Declares which external facts can actually be collected. The EDR stub never claims endpoint visibility or enables writes. |
| sweeps | triage interval/window/caps, hunt interval/window/overlap/limit, watch baseline/forget/limits | Controls collection cadence and volume. Caps create an explicit coverage boundary. |
| sweeps signals | signal window and per-record limits, Kerberos window and spray threshold | Controls deterministic statistical detectors and their input ceilings. |
| sweeps campaigns | watch_quiet_hours, case_quiet_hours, case_context_days, case_context_limit, correlation debounce, Daylight, online intel | Separates one-off lead expiry from established-case lifecycle and bounds how far—and how many—relevant historical decks may be brought forward. The strongest matches are shown; the complete archive remains intact. |
| sweeps health | board interval/windows, episode cooldown/tool rounds | Controls service-health sampling and episode reasoning. |
| sweeps retention | run, trace, and handled retention days | Balances forensic history and storage. Confirm policy before reducing. |
| notifications | webhook URL, signing secret reference, public Nightwatch URL, timeout | Sends signed state transitions and correct investigation links. |
| SMTP relay, credential references, sender/reply-to, public URL, signup domains | Enables durable queued delivery for verified signup, password recovery, and account notices. | |
| auth | session lifetime, secure cookie, login window/lockout, required-MFA roles, policy version, optional Turnstile | Controls local authentication security, legal-policy enforcement, and deployment-specific bot verification. |
| access | per-user/network limits, manual-action rate, concurrency, queue wait, quiet/demo/export policy | Bounds shared interactive capacity and viewer capabilities. |
| tuning | enabled, minimum occurrences, expiration days | Gates connector-declared source-native tuning candidates. Apply and rollback remain typed human actions. |
| evidence_capture | private directory | Stores approved PCAPs; provision and retain it as sensitive evidence. |
| ui | bind, port, minutes-per-detection | Controls exposure and workload projection. Loopback is the safe default. |
| identity records | users, roles, enabled state, TOTP, recovery codes, active sessions | Database-backed access state managed through People & access and each user’s security panel. |
connector_path | absolute or application-relative path | Pins the evidence-source tool surface Nightwatch is allowed to invoke. |
Prefer *_file or *_env fields. Secret files should be readable only by the service account. The browser editor accepts references but does not resolve or return secret values. Do not store literal credentials in source control, release bundles, support archives, or screenshots.
Each target sweeps independently. Give systems in the same real environment the same reconciliation_scope so cross-system references are possible; isolate unrelated customers or environments with different values. Configure internal CIDRs and domain suffixes explicitly.
Shorter intervals increase API work and can outpace downstream reasoning. Change cadence and queue/record caps together, observe a complete peak period, and check coverage indicators. The administration editor enforces bounded values, including a minimum scheduler poll of 10 seconds.
Nightwatch can begin with one source without treating that source as the product. Connectors publish bounded observations with provenance, freshness, scope, completeness, sensitivity, and native identifiers. The reasoning and case layers consume that common envelope.
| Source family | Status in this distribution | Best authority |
|---|---|---|
| Wire / NDR | Available connector | Observed communications, protocol operations, timing, direction, volume, service behavior, and retained payload evidence. |
| Public network context | Available / opt-in | Ownership, route validity, provider ranges, and exact typed reputation context; never a local verdict. |
| Identity from network protocols | Available where visible | Observed protocol identities and authentication behavior, not full directory or IdP audit state. |
| Endpoint / EDR | Contract only | Processes, files, registry, memory, local users, and containment state. |
| Directory / IdP | Contract only | Authentication decisions, authorization, roles, groups, sessions, and identity risk. |
| Asset / CMDB / IPAM | Contract only | Declared owner, role, criticality, service membership, address, and lifecycle. |
| Cloud, vulnerability, change, and ticketing | Contract only | Control-plane events, exposure, approved intent, maintenance, ownership, and incidents. |
| Human context | Attributed notes | Intent, local knowledge, decisions, and approvals; never relabeled as observed telemetry. |
A new connector is complete only when contract fixtures prove cursor behavior, pagination, timeouts, rate limits, partial results, staleness, schema drift, permission loss, and secret redaction. Product-specific language and credentials belong in the optional connector package, not in the core case experience.
Current boundary: scheduled wire collection uses the executable at connector_path. It advertises JSON tool schemas and receives target credentials through NIGHTWATCH_SOURCE_* environment variables. Optional Python adapters currently serve the evidence planner and context registry; installing one does not replace scheduled wire collection. Both paths are being converged on the same manifest, envelope, health, receipt, and failure contract.
Nightwatch’s hunt rules and statistical signals run before model reasoning. Rules are currently shipped as reviewed source, not uploaded from the browser. Configuration exposes a small set of safe thresholds and resource ceilings.
Add a deterministic rule when the evidence condition is precise, repeatable, cheap enough for every round, and independently testable. Use reasoning only to interpret fired evidence in context. Do not encode a vague “suspiciousness” prompt as a rule.
A future rule-pack format should allow signed, versioned packs with metadata, record contracts, queries, thresholds, entity mappings, severity guidance, fixtures, and resource budgets. Packs should pass doctor validation and a dry run before activation. Arbitrary Python, shell, network calls, model prompts, and write tools should remain outside the pack format.
Installed connectors supply observed network and service evidence. Nightwatch’s context layer adds the local identity, asset, history, ownership, and expectation facts needed to explain what that evidence means here. Deterministic code computes the facts and mismatches; a model explains the smallest defensible case from them.
| Label | What it means |
|---|---|
| Mismatch | Observed and expected values can be compared and differ. That is a fact to explain, not proof of attack. |
| Unknown | Data is absent, stale, ambiguous, capped, or unsafe to compare. Unknown is neither match nor mismatch. |
| Shared infrastructure | The provider, ASN, address, certificate, resolver, CDN, or SaaS serves many tenants. It does not identify the actor or purpose. |
| Reputation | A time-bounded third-party report or score. Supporting context, not local observation or a verdict. |
| Proof | Only the narrow claim an authoritative source or observed mechanism supports. A valid RPKI origin does not prove benign traffic. |
Every context fact carries provenance, observation/fetch time, expiry, confidence, scope, and limitations. Failed public lookup means unknown. Learned history means “usual in the retained baseline,” never “approved.” Human notes remain attributed human context, not telemetry.
CONTEXT_REASONING_GUIDE.md contains the detailed source matrix and privacy rules. REASONING_ENGINE.md is the complete source-to-case reasoning contract.
A Nightwatch skill should define a bounded investigation method: what evidence it accepts, which read-only tools it may use, what it must publish, and how success is tested. Skills should not be unscoped prompt fragments.
id: org.example.identity-triage
version: 1.0.0
accepts: [finding.kerberos, finding.ldap]
tools: [records.search, metrics.query] # read-only allowlist
publisher: publish_investigation
max_tool_rounds: 6
evidence_contract: complete-or-declare-gap
tests: [fixtures/spray.json, fixtures/ordinary-sso.json]
Nightwatch sounds like an experienced analyst working beside the operator: calm, candid, specific, and ready to show the evidence. That voice governs interface copy, generated analysis, deterministic fallbacks, notifications, errors, exports, and documentation.
The voice is human because it uses natural words, makes a clear judgment, and respects the reader’s time. It does not use swagger, snark, synthetic empathy, product praise, corporate filler, or dramatic metaphors. It does not soften harm or manufacture urgency. After Hours and Behind the Rack may keep their atmosphere, but a metaphor never replaces a fact or hides a limitation.
NIGHTWATCH_VOICE.md is the complete Watch Partner editorial guide for vocabulary, before-and-after patterns, uncertainty rules, interface conventions, accessibility, localization, and the release checklist.
Nightwatch is an observable reasoning pipeline. The model is one replaceable specialist near the end, not the pipeline itself. Collection, provenance, normalization, memory, arithmetic, correlation, routing, Daylight, workflow, and audit belong to the harness.
Nightwatch resolves, computes, or fetches every relevant fact it is already allowed to obtain before asking the analyst. It reuses fresh case evidence, performs deterministic joins and comparisons, then makes the narrowest useful read-only connector or approved public lookup. Only genuinely inaccessible evidence, human intent, business approval, off-source events, or protected action authority should come back as a question.
Evidence is the versioned fact package from connectors, case memory, local history, public facts, and attributed human knowledge. Harness is Nightwatch: source availability, collection, normalization, caches, permissions, budgets, memory, validation, Daylight, workflow, approvals, and audit. Model is a replaceable specialist that interprets the ambiguity left in the bounded package. A model can explain evidence; it cannot grant itself a source or a write.
The dashboard’s At the Chalkboard room teaches this visually. The repository’s REASONING_ENGINE.md is the normative reasoning and Daylight contract; EVIDENCE_CONNECTORS.md defines source authority and the evidence envelope.
A proper question names the exact missing fact, why Nightwatch could not get it, which source or person can answer, and what each answer changes. “Was change CHG-1842 meant to authorize Aaron’s account on BUILD-04 at 02:10 UTC?” is useful. “Is this expected?” simply hands the investigation back to the human.
| Capability | Status | Important limit |
|---|---|---|
| Case package, memory, notes, and routed evidence-source queries | Current | Tool use is model-selected within bounded rounds; universal proof of exhaustion is still incomplete. |
| Deterministic methods, correlation, material-change gate, and model accounting | Current | Coverage is only as complete as installed sources and method contracts. |
| Separate Daylight pass with fallback and cooldown | Current | Unavailable Daylight leaves the first conclusion authoritative and the gap visible. |
| Public IP/prefix/origin/owner/RPKI and opt-in reputation | Current | Ownership, route validity, and reputation do not prove traffic purpose. |
| Endpoint, directory, IdP, CMDB, cloud, vulnerability, and change connectors | Contract only | No installed connector means Nightwatch must name the gap or ask the right owner. |
| Universal autonomous evidence planner | Registry foundation | The safe executor and proof-of-exhaustion trace remain required. |
| Control | Safe shape | Required proof |
|---|---|---|
| Method routing | Choose a qualified model profile per method and severity band. | Tool-call, schema, latency, cost, and labeled-quality evaluation. |
| Evidence budget | Record/tool/token ceilings with explicit incomplete coverage. | No hidden truncation; stable behavior at each cap. |
| Decision policy | Confidence thresholds for watch, escalate, or request more evidence. | Precision/recall and reopening analysis on labeled cases. |
| Daylight policy | Enablement and method-specific challenge strength. | Ordinary-explanation capture without suppressing real incidents. |
| Context policy | Which prior judgments and tool results may be reused. | Freshness, tenant isolation, and stale-context tests. |
| Voice | Small reviewed presentation profiles, separate from evidence policy. | No change to severity, uncertainty, citations, or safety gates. |
Tool permissions, write authority, publisher schemas, evidence citation rules, secret access, tenant reconciliation, and human approval gates must remain enforced contracts. A custom prompt must never widen them.
Lab exercises can test whether Nightwatch helps a person solve the case without teaching the system the answer. Nightwatch investigates blind; an evaluator freezes the case snapshot; only then does an authorized person reveal a separately stored exercise key. The key records expected campaigns, stages, entities, distractors, known limitations, and the useful action.
The answer key must remain outside the ordinary Nightwatch database and unavailable to schedulers, correlators, reasoning routes, tools, prompts, and the operator board. Saying “this is one campaign” before the snapshot would contaminate the test. After reveal, the evaluator may score evidence and stage coverage, grouping and fragmentation, contamination, story quality, prioritization, and action quality.
| Symptom | Check first |
|---|---|
| No first round | Run doctor; inspect service logs, target auth, scheduler state, and time. |
| Empty board | Check telemetry freshness, target scope, record contracts, and coverage/cap messages. |
| Reasoning backlog | Check primary quota, request spacing, queue limit, fallback eligibility, and repeated unchanged evidence. |
| Wrong entity relationships | Audit internal networks/domains and reconciliation scopes before changing prompts. |
| Webhook missing | Verify URL, secret reference, timeout, receiver signature validation, and transition eligibility. |
| Configuration button denied | Confirm the account has the Admin role and an active authenticated session. |
The administration workspace exposes implemented controls and reserves clearly labeled locations for future safe controls, while keeping evidence contracts and authority fixed. This is the proposed order of work.
| Layer | Configuration direction | Status |
|---|---|---|
| Environment | Targets, identity boundaries, lamps, credentials, storage, notifications. | Current |
| Resources | Cadence, windows, caps, retention, model routes, budgets. | Current |
| Attention | Suppression scope, tuning eligibility/expiry, severity and notification policy. | Partial |
| Detection | Validated versioned rule packs and safe threshold overrides. | Planned |
| Investigation | Signed bounded skills with schemas, tool allowlists, and tests. | Planned |
| Enrollment & identity | Invitation lifecycle, verified SMTP signup, recovery, sessions, TOTP MFA, role policy, and durable mail queue; external identity providers later. | Current |
| Reasoning | Evaluated per-method profiles, evidence budgets, and rollout controls. | Planned |
| Presentation | Adaptive Meaning / Reasoning / Source case composition, work anchors, and validated visuals. | Partial |
| Enrichment | Local identity and asset history first; cached public ownership and vulnerability facts second; optional reputation last. | Partial |
| Lab evaluation | Blind snapshot, isolated sealed answer key, post-reveal scoring, and analyst debrief. | Offline library |
| Authority | Tool permissions, tenant isolation, publisher contracts, typed approvals. | Fixed contract |
Documentation version v2026.08.12. When product behavior and this site disagree, treat that as a defect: verify against the example configuration and release tests, then update both in the same change.