Skip to content

Commit

Permalink
Making Queues Sendable (#129)
Browse files Browse the repository at this point in the history
* The usual package tidying

* First pass at making Queues properly Sendable. Known to be incomplete.

* Some general code cleanup

* More package tidying

* Sendable-ness

* Revamp QueuesCommand to do signal handling correctly, use structured logging, always cancel tasks even if they error, and never deadlock in job scheduling.

* Remove unnecessary overloads from AsyncJob

* Use makeFutureWithTask() and makeSucceededVoidFuture() consistently

* Add utilities for firing notification hooks

* Add some utilities to JobData

* Package cleanup

* Completely redo QueueWorker to be most async in implementation, do structured logging, do much better logging, check delays properly, clear data from unregistered jobs, count remaining attempts properly (and without resetting maxRetryCount every time), and always requeue for retries, and just generally be better.

* Log instead of printing in RepeatedTask+Cancel

* Lots of structured logging

* Lots of general code cleanup and doc comments

* Add `AsyncQueue` to allow creating queues drivers that use async methods.

* Make all the tests fully async

* Address PR feedback - Application.Queues.Storage needs to be actually Sendable-safe, use assert instead of precondition in JobData init, update queuedAt value for retries in QueueWorker, QueuesCommand doesn't need to be @unchcked, make QueuesDriver require `Sendable`

* Update outdated doc comment

* Add some missing tests, add AsyncTestQueueDriver to XCTQueues, respect LOG_LEVEL env var in tests
  • Loading branch information
gwynne authored Jun 13, 2024
1 parent 9117f54 commit 15829c5
Show file tree
Hide file tree
Showing 36 changed files with 1,588 additions and 1,321 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:

jobs:
unit-tests:
uses: vapor/ci/.github/workflows/run-unit-tests.yml@reusable-workflows
uses: vapor/ci/.github/workflows/run-unit-tests.yml@main
with:
with_coverage: true
with_tsan: false
secrets: inherit
45 changes: 31 additions & 14 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,29 +6,46 @@ let package = Package(
platforms: [
.macOS(.v10_15),
.iOS(.v13),
.tvOS(.v13),
.watchOS(.v6),
.tvOS(.v13),
],
products: [
.library(name: "Queues", targets: ["Queues"]),
.library(name: "XCTQueues", targets: ["XCTQueues"])
],
dependencies: [
.package(url: "https://github.com/vapor/vapor.git", from: "4.101.1"),
.package(url: "https://github.com/apple/swift-nio.git", from: "2.53.0"),
.package(url: "https://github.com/apple/swift-nio.git", from: "2.65.0"),
],
targets: [
.target(name: "Queues", dependencies: [
.product(name: "Vapor", package: "vapor"),
.product(name: "NIOCore", package: "swift-nio"),
]),
.target(name: "XCTQueues", dependencies: [
.target(name: "Queues")
]),
.testTarget(name: "QueuesTests", dependencies: [
.target(name: "Queues"),
.product(name: "XCTVapor", package: "vapor"),
.target(name: "XCTQueues")
]),
.target(
name: "Queues",
dependencies: [
.product(name: "Vapor", package: "vapor"),
.product(name: "NIOCore", package: "swift-nio"),
],
swiftSettings: swiftSettings
),
.target(
name: "XCTQueues",
dependencies: [
.target(name: "Queues"),
],
swiftSettings: swiftSettings
),
.testTarget(
name: "QueuesTests",
dependencies: [
.target(name: "Queues"),
.target(name: "XCTQueues"),
.product(name: "XCTVapor", package: "vapor"),
],
swiftSettings: swiftSettings
),
]
)

var swiftSettings: [SwiftSetting] { [
.enableUpcomingFeature("ForwardTrailingClosures"),
.enableUpcomingFeature("ConciseMagicFile"),
] }
54 changes: 54 additions & 0 deletions [email protected]
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
// swift-tools-version:5.9
import PackageDescription

let package = Package(
name: "queues",
platforms: [
.macOS(.v10_15),
.iOS(.v13),
.watchOS(.v6),
.tvOS(.v13),
],
products: [
.library(name: "Queues", targets: ["Queues"]),
.library(name: "XCTQueues", targets: ["XCTQueues"])
],
dependencies: [
.package(url: "https://github.com/vapor/vapor.git", from: "4.101.1"),
.package(url: "https://github.com/apple/swift-nio.git", from: "2.65.0"),
],
targets: [
.target(
name: "Queues",
dependencies: [
.product(name: "Vapor", package: "vapor"),
.product(name: "NIOCore", package: "swift-nio"),
],
swiftSettings: swiftSettings
),
.target(
name: "XCTQueues",
dependencies: [
.target(name: "Queues"),
],
swiftSettings: swiftSettings
),
.testTarget(
name: "QueuesTests",
dependencies: [
.target(name: "Queues"),
.target(name: "XCTQueues"),
.product(name: "XCTVapor", package: "vapor"),
],
swiftSettings: swiftSettings
),
]
)

var swiftSettings: [SwiftSetting] { [
.enableUpcomingFeature("ForwardTrailingClosures"),
.enableUpcomingFeature("ExistentialAny"),
.enableUpcomingFeature("ConciseMagicFile"),
.enableUpcomingFeature("DisableOutwardActorInference"),
.enableExperimentalFeature("StrictConcurrency=complete"),
] }
39 changes: 14 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,18 @@
<p align="center">
<img
src="https://user-images.githubusercontent.com/1342803/75701575-03f34580-5c82-11ea-9960-f39cdd3c8862.png"
height="64"
alt="Queues"
>
<br>
<br>
<a href="https://docs.vapor.codes/4.0/">
<img src="http://img.shields.io/badge/read_the-docs-2196f3.svg" alt="Docs">
</a>
<a href="http://vapor.team">
<img src="https://img.shields.io/discord/431917998102675485.svg" alt="Team Chat">
</a>
<a href="LICENSE">
<img src="http://img.shields.io/badge/license-MIT-brightgreen.svg" alt="MIT License">
</a>
<a href="https://github.com/vapor/queues/actions/workflows/test.yml">
<img src="https://github.com/vapor/queues/actions/workflows/test.yml/badge.svg?event=push" alt="Continuous Integration">
</a>
<a href="https://swift.org">
<img src="http://img.shields.io/badge/swift-5.6-brightgreen.svg" alt="Swift 5.6">
</a>
<a href="https://swift.org">
<img src="http://img.shields.io/badge/swift-5.8-brightgreen.svg" alt="Swift 5.8">
</a>
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/vapor/queues/assets/1130717/40decf1b-bd9e-4347-99ab-d7e27d60f992">
<source media="(prefers-color-scheme: light)" srcset="https://github.com/vapor/queues/assets/1130717/86b36603-bf12-4d0e-8598-45c53caf9608">
<img src="https://github.com/vapor/queues/assets/1130717/86b36603-bf12-4d0e-8598-45c53caf9608" height="96" alt="Queues">
</picture>
<br>
<br>
<a href="https://docs.vapor.codes/4.0/"><img src="https://design.vapor.codes/images/readthedocs.svg" alt="Documentation"></a>
<a href="https://discord.gg/vapor"><img src="https://design.vapor.codes/images/discordchat.svg" alt="Team Chat"></a>
<a href="LICENSE"><img src="https://design.vapor.codes/images/mitlicense.svg" alt="MIT License"></a>
<a href="https://github.com/vapor/queues/actions/workflows/test.yml"><img src="https://img.shields.io/github/actions/workflow/status/vapor/queues/test.yml?event=push&style=plastic&logo=github&label=tests&logoColor=%23ccc" alt="Continuous Integration"></a>
<a href="https://codecov.io/github/vapor/queues"><img src="https://img.shields.io/codecov/c/github/vapor/queues?style=plastic&logo=codecov&label=codecov"></a>
<a href="https://swift.org"><img src="https://design.vapor.codes/images/swift58up.svg" alt="Swift 5.8+"></a>
</p>

<br>

Loading

0 comments on commit 15829c5

Please sign in to comment.