Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 30 additions & 4 deletions docs/book/v2/control-commands.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Available commands and usage

## Summary

Three commands — `failed`, `processed`, `inventory` — report on the queue's logs and
current contents; each is available both via the local CLI and over a TCP message to
the queue server.

## Details

The commands available are:

1. `GetFailedMessagesCommand.php (failed)` - returns logs with messages that failed to process (levelName:error)
Expand All @@ -13,11 +21,11 @@ The commands can be run in two different ways:
To run the commands via CLI, use the following syntax:

```shell
php bin/cli.php failed --start="yyyy-mm-dd" --end="yyyy-mm-dd" --limit=int
php bin/cli.php failed --start="yyyy-mm-dd[ HH:ii:ss]" --end="yyyy-mm-dd[ HH:ii:ss]" --limit=int
```

```shell
php bin/cli.php processed --start="yyyy-mm-dd" --end="yyyy-mm-dd" --limit=int
php bin/cli.php processed --start="yyyy-mm-dd[ HH:ii:ss]" --end="yyyy-mm-dd[ HH:ii:ss]" --limit=int
```

```shell
Expand All @@ -29,11 +37,11 @@ php bin/cli.php inventory
To use commands using TCP messages, the following messages can be used:

```shell
echo "failed --start=yyyy-mm-dd --end=yyyy-mm-dd --limit=days" | socat -t1 - TCP:host:port
echo "failed --start=yyyy-mm-dd[ HH:ii:ss] --end=yyyy-mm-dd[ HH:ii:ss] --limit=days" | socat -t1 - TCP:host:port
```

```shell
echo "processed --start=yyyy-mm-dd --end=yyyy-mm-dd --limit=days" | socat -t1 - TCP:host:port
echo "processed --start=yyyy-mm-dd[ HH:ii:ss] --end=yyyy-mm-dd[ HH:ii:ss] --limit=days" | socat -t1 - TCP:host:port
```

In both cases, the flags are optional. Keep in mind if both `start` and `end` are set, `limit` will not be applied, it's only used when one of `start` or `end` is missing.
Expand All @@ -49,3 +57,21 @@ echo "control" | socat -t1 - TCP:host:port
```shell
echo "inventory" | socat -t1 - TCP:host:port
```

## FAQ

**Q: What's the difference between the `failed` and `processed` commands?**

A: `failed` returns log entries at `levelName:error` (messages that failed to
process); `processed` returns entries at `levelName:info` (messages that processed
successfully).

**Q: Can I filter by date and also cap the number of days?**

A: Yes, but not at the same time — `--limit` is only applied when exactly one of
`--start` or `--end` is given; if both are set, `--limit` is ignored.

**Q: How do I quickly verify the queue is processing messages end to end?**

A: Send the `control` message (e.g. `echo "control" | socat -t1 - TCP:host:port`); it
is always logged as processed successfully, giving you a fast round-trip check.
31 changes: 31 additions & 0 deletions docs/book/v2/how-to/communication-with-queue.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# COMMUNICATE WITH QUEUE

## Summary

Two ways to send a message to Dotkernel Queue from your application: a quick
procedural TCP call, or a reusable service class wired into a Core module.

## Details

Communication with the [`Dotkernel Queue`](https://github.com/dotkernel/queue) can be achieved in two different ways: procedural and object-oriented.

## Procedural approach
Expand Down Expand Up @@ -204,3 +211,27 @@ Navigate to your handler, inject the new service and use your custom method wher
protected NotificationService $notificationService
) {
```

> **_NOTE:_** Sending a message only queues it. `src/App/Message/MessageHandler.php`
> only acts on the literal payload values `control` and `retry` out of the box — add
> your own `elseif` branch (or replace the handler) to process the payload your
> service sends, or it will be consumed silently with no effect.

## FAQ

**Q: Which approach should I use — procedural or object-oriented?**

A: Procedural is simplest for a one-off call; the object-oriented
`NotificationService` approach is better once you're sending messages from multiple
places, since it's reusable and easier to maintain.

**Q: Why does my message need to end with a newline?**

A: The Swoole listener uses the newline as the end-of-message marker; without it, the
server keeps waiting for more data and never processes what was sent.

**Q: My message was accepted but nothing happened — why?**

A: Queuing a message only stores it. `src/App/Message/MessageHandler.php` only has
explicit handling for the literal payload values `control` and `retry` out of the
box; anything else needs a handler branch you write yourself.
29 changes: 28 additions & 1 deletion docs/book/v2/how-to/send-emails.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
# SEND EMAILS

## Summary

How to add background email sending to a Dotkernel application by importing Core
into the queue and following the `send-email` branch as a reference implementation.

## Details

Using a queuing service solves problems such as server overload. For example, if a server receives a large number of requests, it tries to process them synchronously, resulting in long response times or even server crashes.

A concrete example is sending emails. While a series of tasks are running on the server, a task such as sending an email is passed to a queue and run in the background so the server can move on to the next task, while the queue composes the email and sends it. Tasks are queued and processed gradually (FIFO), depending on available resources.

To implement such a service, the [`send-email`](https://github.com/dotkernel/queue/tree/send-email) branch can be taken as a model.

> **_NOTE:_** The default branch 1.0 holds only the base code of Queue and provides essential features such as:
> **_NOTE:_** The default branch holds only the base code of Queue and provides essential features such as:
>
> * Adding messages to the queue
> * Retrieving and processing messages (FIFO)
Expand Down Expand Up @@ -108,3 +115,23 @@ Inside your `config/autoload` folder create a new file named `mail.global.php`,
Once everything is installed and configured we can move on to handle the data in the queue. In the message handler for example `MessageHandler`, each message from the queue is processed, the email is composed, and then sent. By injecting the required services and using templates, the handler can send emails without blocking the main application, respecting FIFO and asynchronous processing.

In this [file](https://github.com/dotkernel/queue/blob/send-email/src/App/Message/MessageHandler.php) you can follow a simple example of how to create and send an email using data received from the queue inside the handler.

## FAQ

**Q: Do I need to modify the base queue code to send emails?**

A: No — import the `Core` module (copied in or as a submodule) and follow the
`send-email` branch as a model; the default branch already provides message queuing
and FIFO processing.

**Q: What does importing Core actually give the queue?**

A: Access to the main application's entities, services and configuration (cache,
mail, authentication, etc.), so the worker can compose and send real emails using
your existing templates.

**Q: Where do I configure the mailer itself?**

A: Create `config/autoload/mail.global.php` from the example in the `send-email`
branch and fill in your mail settings; the queue uses this to send emails in the
background.
40 changes: 35 additions & 5 deletions docs/book/v2/installation.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# INSTALLATION

## Summary

How to get a working copy of the queue running on the server prepared in
[Server setup](server-setup.md): clone the repository, configure
`config/autoload`, install dependencies, register the systemd daemons, and confirm
the listener responds.

## Location

- Because you are logged in now with the non-root user `dotkernel`, your current server path must be `/home/dotkernel`
Expand All @@ -9,15 +16,15 @@
## git clone

```shell
git clone -b default-queue https://github.com/dotkernel/queue.git
git clone https://github.com/dotkernel/queue.git
```

> The installation path should be now `/home/dotkernel/queue`

## Prepare `config/autoload` files

- duplicate `local.php.dist` as `local.php`, then fill in the database credentials and set the `$baseUrl`
- duplicate `log.local.dist` as `log.local`
- duplicate `log.local.php.dist` as `log.local.php`
- duplicate `messenger.local.php.dist` as `messenger.local.php`
- duplicate `swoole.local.php.dist` as `swoole.local.php`

Expand Down Expand Up @@ -55,7 +62,7 @@ sudo systemctl start swoole.service
```

```shell
sudo systemctl status swoole.service
sudo systemctl status swoole.service
```

## Start the Messenger daemon
Expand All @@ -73,13 +80,36 @@ sudo systemctl start messenger.service
```

```shell
sudo systemctl status messenger.service
sudo systemctl status messenger.service
```

### Testing the installation

Send a request from your local machine

```shell
echo "Hello" | socat -T1 - TCP:SERVER-IP:8556`
echo "Hello" | socat -T1 - TCP:SERVER-IP:8556
```

> **_NOTE:_** Any message that is not one of `failed`, `processed` or `inventory` is
> queued twice by design: once with your payload, and once more with the literal
> payload `with 5 seconds delay`, queued 5 seconds later. Expect two entries in
> `inventory`/the logs for every test message you send.

## FAQ

**Q: Which branch should I clone?**

A: Clone without specifying `-b`; this checks out the repository's default branch
instead of pinning to a branch name that can go stale.

**Q: Why does copying `log.local.php.dist` correctly matter?**

A: `config/config.php` only loads local config files that end in `.php`; if the copy
is misnamed the logger silently never loads.

**Q: What should I see after the smoke test?**

A: Two entries appear for the single `echo "Hello"` message you sent — your message,
plus a second, hardcoded `with 5 seconds delay` message queued automatically 5
seconds later.
43 changes: 42 additions & 1 deletion docs/book/v2/messenger-configuration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Messenger Configuration

## Summary

Reference for `config/autoload/messenger.local.php`: the Redis transports Symfony
Messenger uses for new and failed messages, their retry strategy, and the two
message streams (`messages`, `failed`) they map to.

## Details

```php
return [
'symfony' => [
Expand Down Expand Up @@ -35,7 +43,7 @@ return [
],
],
'dependencies' => [
'factories'> [
'factories' => [
'redis_transport' => [TransportFactory::class, 'redis_transport'],
'failed' => [TransportFactory::class, 'failed'],
SymfonySerializer::class => fn(ContainerInterface $container) => new PhpSerializer(),
Expand All @@ -51,3 +59,36 @@ return [
## Dead Letter Queue (DLQ)

DLQ is a dedicated transport where messages are sent when they fail to be processed after a configured number of retries. Each transport can define a retry_strategy specifying the maximum number of retry attempts, delays between retries, and exponential backoff rules. When a message exceeds the allowed retries, it is automatically forwarded to the failure transport and stored in `failed` stream, ensuring that failed messages do not block the queue.

## Application-level retry delays (`fail-safe`)

`config/autoload/local.php` also defines a separate `fail-safe` schedule, used to
delay re-adding a failed message to the queue:

```php
'fail-safe' => [
'first_retry' => 3600000, // 1h
'second_retry' => 43200000, // 12h
'third_retry' => 86400000, // 24h
],
```

This is independent of the transport-level `retry_strategy` above.

## FAQ

**Q: Where do the transport-level retry settings live?**

A: In `config/autoload/messenger.local.php`, under
`symfony.messenger.transports.redis_transport.retry_strategy`.

**Q: What happens once `max_retries` is exceeded?**

A: The message is forwarded to the `failed` transport, defined by
`failure_transport`, and stored in the `failed` Redis stream.

**Q: Is `retry_strategy` the only retry configuration in the project?**

A: No — `config/autoload/local.php` also defines an independent `fail-safe` schedule
(`first_retry`, `second_retry`, `third_retry`) for delaying re-queued messages after
a processing error.
28 changes: 27 additions & 1 deletion docs/book/v2/overview.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
# Overview

> [Dotkernel Queue](https://github.com/dotkernel/dot-queue) is a component based on [**Symfony Messenger**](https://github.com/symfony/messenger) that is used to queue asynchronous tasks.
## Summary

Dotkernel Queue is a Symfony Messenger-based component that lets Mezzio and Laminas
applications hand off slow or unreliable work to background workers instead of
processing it inline.

## Details

> [Dotkernel Queue](https://github.com/dotkernel/queue) is a component based on [**Symfony Messenger**](https://github.com/symfony/messenger) that is used to queue asynchronous tasks.
[netglue/laminas-messenger](https://github.com/netglue/laminas-messenger) is an adapter that integrates Symfony Messenger with the [Laminas Service Manager](https://docs.laminas.dev/laminas-servicemanager/) container for Mezzio/Laminas applications.

Some everyday **operations are time-consuming and resource-intensive**, so it's best if they run on separate machines, decoupled from the regular request-response cycle.
Expand All @@ -23,3 +31,21 @@ It allows the main platform to return a response and remain responsive for new r
[![codecov](https://codecov.io/gh/dotkernel/queue/branch/2.0/graph/badge.svg?token=pexSf4wIhc)](https://codecov.io/gh/dotkernel/queue)
[![Qodana](https://github.com/dotkernel/queue/actions/workflows/qodana_code_quality.yml/badge.svg?branch=2.0)](https://github.com/dotkernel/queue/actions/workflows/qodana_code_quality.yml)
[![PHPStan](https://github.com/dotkernel/queue/actions/workflows/static-analysis.yml/badge.svg?branch=2.0)](https://github.com/dotkernel/queue/actions/workflows/static-analysis.yml)

## FAQ

**Q: What is Dotkernel Queue built on?**

A: It's based on Symfony Messenger, integrated into Mezzio/Laminas applications through
the `netglue/laminas-messenger` adapter for the Laminas Service Manager container.

**Q: Why run tasks asynchronously instead of inline?**

A: Time-consuming or resource-intensive operations would otherwise block the
request-response cycle; running them on background workers keeps the main platform
responsive to new requests.

**Q: Where can I find the project's build and license status?**

A: See the badges above, which link to the GitHub issues, forks, stars, license, CI,
code coverage, and static analysis pages for the `dotkernel/queue` repository.
35 changes: 35 additions & 0 deletions docs/book/v2/server-setup.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Server setup

## Summary

Step-by-step instructions for provisioning a fresh AlmaLinux 9/10 server with the
users, PHP runtime, Swoole and Redis/Valkey extensions, and firewall rules the queue
daemon needs.

## Details

The below instructions were tested only on **AlmaLinux 9** or **10**.

*For other operating systems, they need to be adapted accordingly.*
Expand All @@ -20,6 +28,7 @@ dnf update -y

```shell
useradd dotkernel
useradd --system --no-create-home queue
```

```shell
Expand Down Expand Up @@ -54,6 +63,9 @@ sudo dnf install -y https://rpms.remirepo.net/enterprise/remi-release-$(rpm -E %
sudo dnf module enable php:remi-8.5
```

> PHP 8.4 (`php:remi-8.4`) is also supported (`composer.json` allows
> `~8.4.0 || ~8.5.0`); substitute the module version above if you need 8.4.

```shell
sudo dnf install -y php php-cli php-common php-intl
```
Expand Down Expand Up @@ -166,3 +178,26 @@ sudo firewall-cmd --reload
```

> NOW THE SERVER IS READY

## FAQ

**Q: Which operating systems does this guide support?**

A: It was tested on AlmaLinux 9 and 10; other operating systems need the steps
adapted accordingly.

**Q: Which PHP versions can I install?**

A: PHP 8.5 (`php:remi-8.5`) is documented here, and PHP 8.4 (`php:remi-8.4`) is also
supported since `composer.json` allows `~8.4.0 || ~8.5.0`.

**Q: Which system users does the queue need?**

A: A sudo-capable `dotkernel` user for administration, and a `queue` system
user/group, which is what the shipped `swoole.service` and `messenger.service` unit
files run as.

**Q: Is the firewall setup mandatory?**

A: No, but it's recommended — it restricts inbound connections on the queue's TCP
port (8556 by default) to specific source IPs.
Loading
Loading