servd

Dedicated Queue Runner

Craft CMS launches background jobs in response to actions performed by users or on a schedule. These jobs are normally executed whenever a user logs into the control panel using an ajax request to trigger them.

This implementation can lead to problems if the jobs...

• Aren't enqueued by control panel activity, in which case they'll just sit in a pending state until an admin logs in.
• Take a long time to execute.
  
• Consume a significant amount of CPU as they will starve resources from normal user requests.

You can read a little more about these issues here and here.

On our Starter plan, this standard mechanism remains the default, however our Enhanced and Pro plans provide the ability to run Craft's queue completely isolated from user requests and without needing to be triggered by a user navigating around the Craft control panel.

Activating the Dedicated Queue Runner #

If your project is subscribed to a plan that supports it, you can activate the dedicated queue runner on a per-environment basis.

Troubleshooting #

Most of the time your background jobs will work fine without requiring any changes, however in testing we have found that some jobs can have problems.

Jobs assuming they are being run as web requests

Craft has two primary modes of execution: web and console. When handling web requests, the core framework has access to things like session information and the URL that was used to trigger the request. When handling a console request these pieces of information aren't present.

Some background jobs (mainly those from originating from plugins) assume that all jobs will be executed as a web request and therefore try to access session or URL information. This causes the job to throw an error when being executed from a console command (which is how Servd's background jobs are executed).

This will often present itself in the logs as Session does not exist in a console request or Calling unknown method: craft\console\Request::getPathInfo().

You can determine whether or not your jobs might run into problems by setting 'runQueueAutomatically' => false in your project's config/general.php, queuing up some jobs and then executing ./craft queue/run on the command line in your project's root directory.

Jobs assume a single filesystem

Servd runs your queue in isolation to prevent it from interfering with normal user requests. This means that it also has an independent filesystem.

During testing we have found that some jobs assume a single filesystem is being shared between all component parts of the application. Most notably the Feed Me plugin writes all of its logs to the filesystem to later be displayed in the Craft control panel. This is not compatible with a high-availability environment and results in the logs simply not displaying in the control panel. We have therefore added a setting in the Servd Plugin which forces feedme to write its logs to the standard log output so that they can be collected and displayed in the Servd dashboard.

Resolving Problems #

First, look at the logs in the Servd dashboard to see if anything has been logged there. Usually, any errors will be easy to find there.

If you do have problems with a job's compatibility, you can disable the setting either temporarily or permanently which will cause Craft to fall back to processing tasks using web requests on your project's user-facing instances.