ZZZ Digital Lab

Back to n8n workflow hardening

Synthetic n8n hardening handoff

Local verification

Use Node 20+ from this folder:

node tests/run.cjs
node lint.cjs after/main.json after/error-workflow.json
node demo.cjs

Require 29/29 tests, 0 hardened findings and the expected replay outcomes in evidence/replay.json. The baseline linter is expected to exit 1. evidence/lint.json records 14 baseline findings without printing inline values. Baseline PASS tests confirm the defect exists; they do not approve its use.

The retry simulation reads the exported maxTries and waitBetweenTries settings and records intended waits without sleeping. It runs the exact embedded Code-node sources. It is not an n8n execution, timing or durable-storage test.

Client-controlled staging, not performed here

  1. Back up the existing workflow export, instance version, current activation status, relevant failed executions and an owner-approved restore point. Do not export actual credential values into the project.
  2. Import after/error-workflow.json and after/main.json as new, inactive staging workflows. Validate node type versions and syntax in the installed n8n release. If an import or parameter fails, resolve it before any activation.
  3. Create/configure credentials only in the instance. Rebind the webhook header-auth credential and the HTTP header-auth references for delivery, dead-letter and incident sinks. Replace .invalid URLs with client-approved staging services. Never put secrets into workflow JSON.
  4. The import may assign new workflow IDs. Select the actual imported error workflow in the main workflow's Settings. Do not assume the placeholder ID is portable. Set the instance's permitted retention for failed execution data and ensure it does not outlive the client's policy.
  5. Establish the receiver contract: atomic one-payload-per-key writes, same-payload replay without new side effects, conflicting-payload rejection. Establish durable upsert for dead letters and incidents. The included in-memory fake is not a deployable persistence layer.
  6. Following the installed release's Error Trigger instructions, enable/publish only the staging workflows required for an automatic test. Manual editor execution alone cannot verify Error Trigger delivery. The supplied exports are deliberately inactive; this job did not activate or publish anything.
  7. Replay the synthetic fixture locally to staging under client control. Check: 2 unique accepts; repeat adds 0; controlled transient errors respect the retry bound; exhausted delivery persists original event to dead letters and produces a visible failure/incident; failed dead-letter persistence also produces a visible incident; recovery accepts the remaining event once. Inspect original-event linking for multiple failed items.
  8. Confirm request and overall timeouts, webhook acknowledgement behavior, reverse-proxy/client timeout, actual node retry behavior and status-code policy. Native retries may repeat a whole node/batch, so receiver idempotency must be durable and concurrent-safe. Approve production rollout separately.

Recovery

Retain the original envelope and its event IDs. After the downstream issue is fixed, replay using the same idempotency keys. Reconcile the receiving system with the dead-letter entry, then have an operator close the review record. The demo deliberately retains that record after recovery; it does not erase evidence automatically.

If the dead-letter sink is unavailable, the path must fail visibly rather than claim the item was safely queued. Inspect the local failed execution under the agreed retention policy and recover the original event. The incident sink can also fail; no workflow can guarantee delivery to an unavailable external service. Instance-level operations/monitoring require a separate agreement.

Rollback

Stop the staging/new workflow's activation first. Restore the saved workflow export and its prior settings/activation state in a controlled window. Keep newly written dead-letter and receiver records for reconciliation; reverting workflow JSON does not undo remote business writes. Do not delete credentials, all execution history or unrelated workflows. Rebind only the saved workflow's credential references as needed.

For local files, restore the previous export snapshot or regenerate the supplied synthetic exports with python -B build_workflows.py. Never deploy before/broken.json as an operational rollback. No production writes or account changes were performed in this demonstration.

Boundaries

The linter is policy-specific, and the JavaScript replay is a contract simulation. Live n8n import, credentials, activation, retry timing, item linking and Error Trigger operation remain client staging requirements. Review using AI coding tools in this session is not independent human certification.