fix: check backfill table before Redis so it can override stuck queue… (

#937)

* fix: check backfill table before Redis so it can override stuck queued transactions

Previously the backfill lookup only ran when a transaction was missing
from Redis. Orphaned "queued" transactions that still exist in Redis
were unreachable by backfill. Moving the check first lets backfill
entries act as intentional overrides for stuck queue IDs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: check backfill before Redis in get-logs endpoint too

Same change as status endpoint — backfill entries should take priority
over stale Redis data for consistency across all endpoints.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: treat backfill entry as authoritative in get-logs endpoint

When a backfill entry exists (even if errored), skip the Redis lookup
entirely. Previously an errored backfill entry would fall through to
Redis, potentially returning stale data for an orphaned transaction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Feature/tx backfill fallback (#933)

* feat: add transaction backfill fallback for queue ID lookups

- Add ENABLE_TX_BACKFILL_FALLBACK env var (default: false)
- Add backfill methods to TransactionDB (getBackfillHash, setBackfill, bulkSetBackfill)
- Add fallback lookup in /transaction/logs endpoint
- Add POST /admin/backfill endpoint for loading backfill data

When enabled, the /transaction/logs endpoint will check a fallback
Redis table (backfill:<queueId>) when the primary transaction cache
misses. This allows recovering transaction logs for queue IDs that
were pruned from the main cache.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add DELETE /admin/backfill endpoint to clear backfill table

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs: add comments explaining AMEX backfill logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: validate backfill hash format before use

Add isHex validation to prevent unexpected behavior if malformed
data exists in the backfill table.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: extend backfill fallback to support /transaction/status endpoint

- Add BackfillEntry interface with status field ("mined" | "errored")
- Update TransactionDB to store/retrieve JSON format for backfill entries
- Add backfill fallback lookup to /transaction/status routes
- Update /transaction/logs to use new getBackfill method
- Update admin backfill schema to accept status field
- Maintain backwards compatibility for plain string tx hash entries

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs: add AMEX backfill comments to /transaction/status routes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use discriminated union for backfill schema and remove duplicate comment

- Use discriminated union so transactionHash is required for mined entries
- Remove duplicated AMEX comment block

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

fix: retry webhooks on 429 and 403 responses (#931)

Previously, only 5xx errors triggered webhook retries. This change adds
429 (Too Many Requests) and 403 (Forbidden) to the retry logic, treating
them the same as server errors with exponential backoff.

This prevents webhook events from being lost when the destination
endpoint rate-limits requests or returns transient auth errors.

Also fixes a pre-existing lint error (unused template literal).

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Increase webhook retry attempts from 3 to 5 (#927)

* Increase webhook retry attempts from 3 to 8

- Changes webhook delivery attempts from 3 to 8
- Keeps exponential backoff with 5-second base delay
- Total retry window increases from ~15s to ~10m 35s
- Provides better resilience for temporary outages, deployments, and restarts

Timeline:
- Attempt 1: 0s
- Attempt 2: +5s (5s total)
- Attempt 3: +10s (15s total)
- Attempt 4: +20s (35s total)
- Attempt 5: +40s (1m 15s total)
- Attempt 6: +1m 20s (2m 35s total)
- Attempt 7: +2m 40s (5m 15s total)
- Attempt 8: +5m 20s (10m 35s total)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Reduce webhook retry attempts from 8 to 5

- Changes webhook delivery attempts from 8 to 5
- Keeps exponential backoff with 5-second base delay
- Total retry window: ~1 minute 15 seconds
- Balances resilience with resource efficiency

Timeline:
- Attempt 1: 0s
- Attempt 2: +5s (5s total)
- Attempt 3: +10s (15s total)
- Attempt 4: +20s (35s total)
- Attempt 5: +40s (1m 15s total)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

add webhook debugging logs (#924)

feat: add QUEUE_JOB_RETENTION_AGE_SECONDS to environment configuration (

#921)

Introduced a new environment variable QUEUE_JOB_RETENTION_AGE_SECONDS to define the age (in seconds) after which completed or failed jobs are eligible for removal. Updated job options to utilize this variable for job retention settings.

allow cancelling userops (#918)

* allow cancelling userops

* userophash needs to be valid for schema

fix: correct toAddress assignment in transaction queueing logic (#917)

feat: add experimental retry for prepareUserOp errors in transaction …

…processing (#916)

Introduced a new environment variable EXPERIMENTAL__RETRY_PREPARE_USEROP_ERRORS to enable retrying on prepareUserOp errors instead of failing the transaction immediately. Updated transaction handling logic to support this feature.

Update thirdweb to 5.105.42 (#914)