The benefits of background workers.

aka shifting the load off your webserver.


Press 'c' to toggle code style
Press 's' for speaker notes

Server Client
Server Client

Denial of service attack

4GB / 128 MB = 32 requests

Bad user experience

Form 1
Submit form
Form 2
Submit form


CVE-2019-11037 - my bad

  push graphic-context
  viewbox 0 0 640 480
  fill 'url("|ls "-la)'
  pop graphic-context

  // Upload as .mvg file


Server Client Worker Request $jobID $jobID data? Not ready $jobID data? Here's the data

Demo time


  • Job/message queue
  • Some code to send/receive the jobs
  • Something to keep the code running
  • Some code to stop the jobs

Message queues

Redis, Gearman, RabbitMQ, Amazon SQS

  • Very fast
  • Useful for caching
  • Lua scripting embedded

Some code to send the jobs

class ImageJob {

    /** @var string */
    private $text;

    function __construct(string $text) {
        $this->text = $text;

    public function toString(): string {
        $data = ['text' => $this->text];

        return json_encode_safe($data);

    public function getText(): string {
      return $this->text;

    public static function fromString(string $string) {
        $data = json_decode_safe($string);
        return new self($data['text']);

Some code to send the jobs

class RedisImageJobRepo {

    public function queueImageJob(ImageJob $imageJob)
        return true;

Some code to retrieve the jobs

class RedisImageJobRepo
    function waitForImageJob(int $timeout = 5): ?ImageJob {
        $key = ImageJobKey::getAbsoluteKeyName();

        // Next line sits and waits for a job
        $listElement = $this->redis->blpop([$key], $timeout);
        if (count($listElement) === 0) {
            return null;

        $keyReturned = $listElement[0];
        $data = $listElement[1];

        $imageJob = ImageJob::fromString($data);

        return $imageJob;

Code for encapsulating keys

class ImageJobKey {
  static function getAbsoluteKeyName() : string {
    return str_replace('\\', '', __CLASS__);

  static function getKeyNameForStatus(string $jobId): string {
    return str_replace('\\', '', __CLASS__) .
      ':status:' . $jobId;

Something to keep the code running


Supervisord config

command=php cli.php process:image_example

# also boring log file stuff...
Time Features delivered Simpler tech More capable tech Simpler tech wins More capable tech wins You must choose, wisely

"It's Time For Some Game Theory"

Actually, no.

Emails: 4 GB / 16 MB per worker

256 workers
* 8 business hours a day
* 5 days a week
* 4 weeks a month
* 3600 seconds in an hour
/ 4 seconds to send an email

= 36,864,000 emails a month


Code to run in a loop

function continuallyExecuteCallable($callable, int $maxRunTime)
    $startTime = microtime(true);
    while (true) {
        if (checkSignalsForExit()) {

        if ((microtime(true) - $startTime) > $maxRunTime) {

Make code run in a loop

  • Avoid DOS your own system
  • Avoid DOS other people's system
  • Avoid spinning your CPU at 100%

Some code to stop the jobs

aka listen to signals.

  • SIGINT - Ctrl+C
  • SIGTERM - graceful shut down
  • SIGKILL - ungraceful
“Upon the receival of the SIGTERM, each container should start a graceful shutdown of the running application and exit.” “If a container doesn’t terminate within the grace period, a SIGKILL signal will be sent and the container violently terminated.”
function checkSignalsForExit()
    static $initialised = false;
    static $needToExit = false;

    if ($initialised === false) {
        $fnSignalHandler = function ($signalNumber) use (&$needToExit) {
            $needToExit = true;

        pcntl_signal(SIGINT, $fnSignalHandler, false);
        pcntl_signal(SIGKILL, $fnSignalHandler, false);
        pcntl_signal(SIGQUIT, $fnSignalHandler, false);
        pcntl_signal(SIGTERM, $fnSignalHandler, false);
        pcntl_signal(SIGHUP, $fnSignalHandler, false);
        pcntl_signal(SIGUSR1, $fnSignalHandler, false);
        $initialised = true;


    return $needToExit;

Other benefits

  • Disable background workers when service is down
  • Retrying failed jobs
  • Performance tuning
  • Performance monitoring and scaling
  • Use different language


Twitter: @MrDanack

The benefits of background workers.

Some things that need to be processed in an web based application can take a long time to run. Moving this work to be done in a background worker can provide lots of benefits for the user and make your life easier as a developer. Dan's going to talk about this stuff, going through exactly what the problems are, how to implement background workers, and the diverse benefits they give.