Growing pains The Problem
Our client’s learning material needed a new home. Their existing site, while functional, was built on fully custom tech, making it difficult and expensive to add new features and grow their platform. No existing vendor offered all the features they needed and building a new custom LMS would be to repeat what got them into this situation in the first place. A WordPress-based LMS seemed like a sensible approach but with hundreds of thousands of users and counting, they needed to make sure their new platform could support their growth well into the future without changing their existing branding and interface for the time being.
Starting fresh The Solution
WordPress and its massive plugin ecosystem are great when it comes to rapid development, but as a platform built for DIY bloggers, many plugins and database architectures just weren’t built to handle the performance needs of enterprise-scale websites. Luckily, this wasn’t our first rodeo. By leveraging our own time-tested tech stack and custom in-house software and plugins, we delivered an LMS that rivals the performance of a custom-built platform with the added flexibility of the open WordPress ecosystem. Below are a few of the pieces that went into building a LearnDash site that scales.
LMS-optimized hosting and caching
The backbone of any web platform is its hosting, and this project was no different. LMS platforms are unique in their need to support high concurrency—that is, the ability to process the data of multiple users simultaneously. At this scale, the demands on the server when processing a student’s quiz results can be significant, which is why we’ve spent the past five years perfecting our own scalable cluster-based hosting architecture. To translate that for non-nerds, that means having several servers ready to process user data along with a device that distributes tasks across all those servers, called a load balancer.
Another common practice to improve a site’s performance involves caching data, in other words saving a pre-built copy of commonly-accessed data to avoid the server needing to fetch and build full web pages each time a page is accessed. In the context of an LMS, this gets complicated due to the massive amount of user-specific data that are being accessed, such as progression, quiz results, and more. We’ve yet to find an off-the-shelf solution that can successfully cache LMS data without serving stale data, so we’ve developed our own Redis object caching system that also runs as a cluster, helping to cut down on server requests while only caching data that doesn’t affect the student’s personalized learning journey.
Custom-built templates replaced bloated plugins and themes
The WordPress plugins used on many LMS builds, such as page builders and social forum tools, are often built with the average website in mind: small e-learning or membership sites built for a few thousand users. But when scaling this software to hundreds of thousands of users, small inefficiencies in a plugin’s code can have massive repercussions.
The first step in optimizing the plugin stack was simple: cutting out unnecessary plugins. While page builders are often required to quickly build dynamic websites, we cut out the page builder entirely on this project by using a lightweight in-house templating tool, called Loops & Logic, alongside WordPress’ built-in page builder, Gutenberg. This allowed our front-end developers to easily build the site’s theme without the performance overhead of a page builder.
Another piece on the chopping block was WordPress’ own user roles engine, used to attribute and check the access permissions of each user. Our testing revealed that the native system for user roles starts causing performance issues with databases of over 10,000 users and at 100,000+ users, the standard user roles engine would cause some pages to take several minutes to load while WordPress looked up role information. We used an alternative system which boosted the performance of the site.
Social learning is a common need for modern e-learning platforms and the ability to engage and interact with students is often key to a strong learning experience. Some LMS platforms will use Buddyboss, which adds social-media-like interactivity directly within the WordPress environment. While this plugin offers plenty of enticing features, the way WordPress stores and fetches data from its databases combined with the Buddyboss’ feature-rich bloat can make simple social threads have massive impacts on a site’s performance at scale.
The simplest way to avoid overloading the WordPress database with social threads is to move social features beyond WordPress. A well-built solution for this is Circle, a SaaS that offers social features and more on a platform that can be fully skinned to match your brand. One caveat to this approach is that Circle isn’t open-source or self-hosted, which means that per-user fees can get pricey at scale. Another downside is that it requires social feeds to be on an entirely separate platform, which can feel disruptive to the learner’s experience compared to comment threads and feeds built into the LMS interface alongside their courses, lessons, and learning plans.
Our approach combined the best of both worlds through deep integration with Discourse, an open-source and self-hosted forum software. All social features like threads and comments were handled by Discourse to keep the WordPress LMS running fast, but a custom-built integration allowed our developers to load up this data directly within WordPress to make it feel as though it was a native part of the course experience without impacting performance.
Finally, to optimize the myriad of other functions and scripts being run by the LMS, we developed a profiling tool in-house to give ourselves deep visibility into the site’s background processes on a per-user basis. This tool helps our team fine-tune on-page actions like quiz submissions that are a challenge to optimize with more general web- or server-based profiling tools.
LMS reporting and management without the slowdowns
A responsive course interface is key to a positive user experience, but no e-learning business can run properly without equally quick management and reporting interfaces. As anyone with a large user base can attest, running sitewide reports on thousands of students—each with thousands of data points—can put a strain on even the most lightweight sites.
To address this, we needed to make sure that large database queries were carried out in such a way as to minimize the impact on those using the site. The first part of this solution involved database clustering technology which would reroute expensive queries to a quarantined database. This avoided large reports from impacting the main databases that were being used to serve the rest of the site. The second part of our approach involved developing a custom asynchronous background processing system for batch operations. For example, if a site administrator needed to move 100,000 students into new classrooms programmatically without crashing the site, our system would allow that processing to be completed in batches and carried out quietly in the background without disturbing students completing their courses.
Maintaining backups is another critical piece of any LMS since losing even a few hours of data risks setting back the progress of thousands of users. For this project, we implemented a custom incremental backup solution. Instead of taking full backups of the site which could quickly result in terabytes of redundant data, our LMS-specific solution intelligently understands course-related progress and updates and only appends new information to existing full backups that are taken periodically. This allows for very frequent backups of enormous sites with minimal performance impact while using very little storage space.
