Greptile Summary

This PR introduces Redis-backed distributed locks to serialize chunk-upload state reads and finalizations across concurrent requests for storage files, function deployments, and site deployments. Chunk writes themselves remain outside the lock, preserving parallel throughput. End-to-end tests using Swoole coroutines verify the parallel-upload path and include retry logic for 429 responses.

Confidence Score: 4/5

Safe to merge with minor follow-up; the lock design is sound but the checkLock TTL configuration in Functions/Sites is slightly inconsistent.

All findings are P2. The core double-finalization race is correctly prevented by the lock + in-lock DB re-read. LockContention now correctly returns 429. Previous P1 concerns from the thread appear addressed in the current code. The only new finding is that checkLock in Functions/Sites uses ttl:120 matching its timeout:120, which is an edge-case concern with no practical impact for a fast DB-read critical section.

Functions/Http/Deployments/Create.php and Sites/Http/Deployments/Create.php — checkLock TTL configuration.

Important Files Changed

_{Reviews (6): Last reviewed commit: "Fix deployment upload analyze warning" | Re-trigger Greptile}

✨ Benchmark results

Comparing 1.9.x (before) to fix/parallel-storage-chunk-upload (after).

Before

After

Delta

🔄 PHP-Retry Summary

Flaky tests detected across commits:

Note: Flaky test results are tracked for the last 5 commits

Addressed the Greptile review feedback:

• Changed lock contention from a 500 to 429 (GENERAL_RATE_LIMIT_EXCEEDED) and added Retry-After: 5.
• Reduced the distributed lock TTL to 120 to match the lock acquisition timeout/commented behavior.
• Fixed the double-finalization race by basing finalization on freshly-read DB progress inside the second lock and only finalizing when the stored upload progress is still incomplete.
• Kept uploaded chunk progress monotonic with max($uploaded, $chunksUploaded).

Verified with:

• php -l src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Create.php
• vendor/bin/pint --test --config pint.json src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Create.php tests/e2e/Services/Storage/StorageBase.php
• docker compose exec appwrite test tests/e2e/Services/Storage --filter='testCreateBucketFile(OutOfOrder|ParallelChunksLargeFile)'
• docker compose exec appwrite test tests/e2e/Services/Storage

Addressed the latest Greptile TTL finding:

• Restored the distributed lock hold TTL to 600 seconds so the lock can outlive long finalization work such as AV scan, compression, encryption, and the DB write.
• Kept the withLock(..., timeout: 120.0) acquisition timeout unchanged so waiting callers still stop after 120 seconds and receive the existing retryable 429 response.
• Also removed the unused mode injection/action parameter from createFile.

Verified with:

• php -l src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Create.php
• vendor/bin/pint --test --config pint.json src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Create.php
• docker compose exec appwrite test tests/e2e/Services/Storage --filter='testCreateBucketFile(OutOfOrder|ParallelChunksLargeFile)'
• docker compose exec appwrite test tests/e2e/Services/Storage

Addressed the latest Greptile review feedback:

• Guarded queueForEvents so the files.create event is only queued when chunksUploaded === chunks, preventing duplicate create events for intermediate chunks.
• Added bounded retry handling for 429 responses in testCreateBucketFileParallelChunksLargeFile, retrying only lock-contention responses and respecting Retry-After when present.

Verified locally with:

• php -l src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Create.php
• php -l tests/e2e/Services/Storage/StorageBase.php
• vendor/bin/pint --test --config pint.json src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Create.php tests/e2e/Services/Storage/StorageBase.php
• docker compose exec appwrite test tests/e2e/Services/Storage --filter='testCreateBucketFile(OutOfOrder|ParallelChunksLargeFile)'
• docker compose exec appwrite test tests/e2e/Services/Storage