Reliability

Sonar assumes a queue message, callback, or finalization attempt can be retried. Its delivery model is at least once; reliability comes from making repeated effects safe and recoverable.

Dispatch is retryable

Each task has a stable dispatch key derived from its scan, task, and step plus a unique attempt ID and callback credential. The worker checks task state before running and reports a started callback before command execution.

If a transient backend or network failure prevents the worker from receiving a durable result disposition, it negatively acknowledges the RabbitMQ message with requeue. The message can return; the backend decides from persisted task state whether work should run or be skipped.

Callbacks are authenticated and idempotent

Started and result callbacks are signed with a secret created for that dispatch attempt. The backend validates the attempt, stable dispatch identity, worker name, output digest, and HMAC.

The task state machine accepts only legal transitions. Repeating the same accepted result digest returns a duplicate disposition without enqueuing another effect; a different result for an already accepted task is a conflict.

This prevents a duplicate callback from silently replacing the result already chosen for a task.

Fan-out is idempotent

For Single dependencies, the upstream task ID is the downstream child's ParentTaskId. The reconciler searches for completed parents that do not already have a child. Repeating that query after a restart or overlapping trigger therefore converges on one child per upstream task.

See Dependencies and data flow for the Single and All rules.

Result effects are idempotent

Accepting a task result creates durable finalization work keyed by task and result digest. Leased workers may retry that work after interruption. Structured output uses schema-specific conflict keys and database upserts, including in-batch deduplication, so replaying an observation does not create another logical asset.

Idempotency is scoped to declared identities. If a tool changes the value that defines an asset or the workflow targets a different table, Sonar correctly treats that as a different effect.

Failure is explicit

• A command failure moves its task to Failed rather than leaving it in progress.
• Durable finalization retries with backoff; repeated poison work is retained with an error and the task becomes failed.
• A scan whose step has terminal tasks but no completed task reaches CompletedWithErrors rather than remaining open indefinitely.
• Empty file or line expansion produces an explicit no-work terminal result.

Recovery of work whose machine disappears is documented separately under Recovery and capacity.