servd

For most developers, PHP sessions just work, and we don't need to worry too much about how they work under the hood. But occasionally they throw up issues which require us to have a look under the hood and understand exactly what's going on. On some of these occasions, we might even find an interesting quirk that needs fixing.

One such occasion happened to us recently whilst investigating a problem with disappearing sessions in a Craft CMS project. The project itself had been working fine up until the point that a plugin was activated which stored additional login context (2FA) in the PHP session. Once this plugin had been enabled, Craft CP users began being logged out regularly, even whilst in the middle of performing a series of tasks.

To track this down, we had to dive pretty deep into the codebases for both Craft and Yii, to figure out exactly what was going wrong. But first, a primer.

How PHP Sessions Work

PHP sessions are an opt-in piece of functionality. They are only triggered when you call the session_start() function within your PHP codebase. When this function is called, PHP does several things:

1. Opens/initialises the session
2. Finds any existing session identifiers stored in cookies sent along with the request
   
3. Reads in any existing session data, from session storage, to the $_SESSION array
4. Waits for the request to finish or for session_write_close() to be called
5. If the session data has changed in any way, writes the $_SESSION array back into session storage
6. If the session data has not changed in any way, 'touch' the session storage object to update its last used timestamp
7. Garbage collection
   

This sequence of events allows PHP to read, write and keep track of the last access time for any individual session.

Another important part of the PHP sessions system is the garbage collector. This is a background process which cleans up any session data that is no longer required because it is too old. We can configure how old 'too old' is by tweaking the session.gc_maxlifetime parameter within PHP's configuration. If the garbage collector finds a session object which has not been used for at least this long, it gets deleted.

This all seems sensible and for the most part, works as expected. A session is maintained until the user has a period of inactivity which is at least as long as session.gc_maxlifetime, at which point the session is deleted from storage.

PHP Session Handlers

Although the overall process for handling sessions within PHP is always the same, the storage mechanism that we use for the data can change depending on our circumstances. By default, PHP uses Filesystem Session Storage, in which session objects are stored as individual files on the filesystem and their last used timestamps are stored as the 'modified' time on those files.

When working in a load balanced environment, like Servd, we need a different solution because each load balanced component has its own filesystem, so we'd end up with sessions on some instances and not others. We can therefore pull the sessions out to an external storage. In our case, we use Redis.

To swap out the storage mechanism used by PHP for its session data we need to provide it with a custom Session Handler. This is a class which implements all of the functionality needed for the 6 steps listed above. Because we're using Craft, and therefore Yii under the hood, we use Yii's official yii2-redis session handler.

Back to Our Problem Project

So now that we know how sessions are working, can we track down exactly what's going wrong with our disappearing sessions?

What we knew so far:

• After a low-ish number of minutes (10-30) after the user logged into the Craft CP, they'd be kicked out to the login page
• The problem only happened if a 2FA plugin was active
• The 2FA plugin was well-used and updated, so likely wasn't the root cause unless it was conflicting with something else

So the problem felt like it was session-based, but why was it only occurring when the 2FA plugin was active?

Craft's Session Backup Strategy

In order to try to work around some of the limitations of PHP's built-in session management solution, Craft/Yii have their own abstraction built on top of it. Whenever a user authenticates, a database record is created in a table called sessions which tracks the user's login state. This is then linked up to another auth cookie which contains a secret value used to match users to their database-stored session info.

So logged in users actually have two sessions, one handled by PHP, and another handled directly by Craft/Yii.

This allows Craft to define its own configuration for session lengths, irrespective of PHP's config settings. When a request arrives which does not have an existing PHP session, Craft can check to see if there's self-managed session row in the database. If so, it restores the PHP session with the state found in the database. Even if a user's PHP session is lost, Craft is able to restore it from the database! Pretty handy!

However, the database-held session does not contain all of the data that we might write to $_SESSION, only a subset which Craft/Yii are interested in. So if we do lose our PHP session, we can only partially restore the user's previous session.

Can you see where we're headed here?

Back to Our Problem Project (Again)

Under normal conditions, if the PHP session gets lost, then Craft is able to automatically log the user back in without missing a beat. But when the 2FA plugin is active on our project, which stores its state within the $_SESSION array, Craft will still log the user back in, but without any of the 2FA auth information present. That immediately causes the 2FA plugin to kick the user back out to the login screen for being sneaky.

Ok, so now that we've figured that out, why are we losing PHP sessions unexpectedly?

This took a bit of digging and ultimately we tracked it down by comparing the File System Session Handler to Yii's Redis Session Handler. Here's what each of them does at each step of the PHP session process:

1. Opens/initialises the session
   
   Filesystem Handler: Ensures the directory where session files are stored exists and is wrtable.
   
   Redis Handler: Checks that the connection to Redis works
   
   
2. Finds any existing session identifiers stored in cookies sent along with the request
   
   Same for both: checks the php cookie session cookie and extracts the session identifier
   
   
3. Reads in any existing session data, from session storage, to the $_SESSION array
   
   Filesystem Handler: Reads the contents of a file which has a filename matching the session identifier
   
   Redis Handler: Runs a GET request to obtain the value of the key matching the session identifier
   
4. Waits for the request to finish or for session_write_close() to be called
   
   Same for both
   
5. If the session data has changed in any way, writes the $_SESSION array back into session storage
   
   Filesystem Handler: Writes the new $_SESSION content out to a file which has a filename matching the session identifier. This also updates the file's last modified timestamp.
   
   Redis Handler: Runs a SET to store the new $_SESSION content to a key which matches the session identifier. Sets the Redis TTL of the key to the current time + session.gc_maxlifetime
   
   
6. If the session data has not changed in any way, 'touch' the session storage object to update its last used timestamp
   
   Filesystem Handler: Runs touch on the file which has a filename matching the session identifier in order to update the file's last modified timestamp.
   
   Redis Handler: Does nothing (!!!!) 🚨
   
   
7. Garbage Collection
   
   Filesystem Handler: Deletes any session files with a last modified time < now - session.gc_maxlifetime
   
   Redis Handler: Does nothing, the native Redis TTLs take care of cleaning up expired keys.
   

You can probably tell from the added emphasis where the problem lies. With the Redis Session Handler, if a session's data isn't changed, its TTL is never updated, causing it to be cleaned up by Redis after session.gc_maxlifetime, even if it is regularly being accessed.

We've found the root cause of our problem - a missing piece of functionality in yii2-redis' session handler.

Fixing It

There are three ways to fix this thing:

1. Add the correct functionality to the yii2-redis package. We'll work up a patch and get it submitted as a PR unless we can find a specific reason why this has not already been fixed. It'll likely take a while to get approved.

2. Change PHP's session.lazy_write config option to 'Off'. This forces PHP to write the entire $_SESSION array back to storage at the end of every request even if it hasn't changed. This would fix the issue because it would essentially eliminate the problematic step 6 in the list above, but if you have a non trivial amount of data in your session, it could hurt performance on every page request.

3. Use a custom, fixed Session Handler which updates the Redis TTL for the session on every access.

We've opted for option 3, by including a new Session Handler in the Servd Plugin that extends from the original yii2-redis one, but adds a simple fix.

We then inject a little bit of additional configuration code into your Craft projects during bundle builds which checks to see if this custom Session Handler is available and if it is, it'll use it.

All projects running on Servd with the latest version of the Servd plugin installed will benefit from this fix and won't see any related session problems (which could also impact Craft Commerce projects under certain circumstances).