Benchmarks

Hammer on a product catalog page

I tested on an otherwise idle WooCommerce / Bookshop staging site with 1,350 products. A benchmark of product catalog page 75 out of 76 showed this number of pageviews per second, higher is better.

• 13.2 with no persistent object cache.
• 18.0 with this SQLite Object Cache (36% more than no cache).
• 15.0 with the Redis Object Cache running on localhost (25% more than no cache).
• 11.3 with the Redis Object Cache running on a dedicated machine (14% less than no cache).

These benchmarks were run on Ubuntu 22.10, WordPress 6.0.1, php 8.0.27, and SQLite 3.37.2, php-fpm 8.1, redis 6.0.16, and Apache 2.4.54. I used ab, the Apache HTTP Server Benchmarking Tool.

Benchmarking work is ongoing. If you run benchmarks please post the results in the support forum.

It’s surprising that using redis on its own dedicated machine slows things down rather than speeding them up. It’s possible I’m not measuring things correctly. And this is surely a distorted test workload. Again, benchmarking work is ongoing.

Load test with many transients

I created an object cache load test. It works by setting a large number of transients, then getting them, then deleting them. Transients go into the persistent object cache rather than the site database if a cache is provisioned on the site, so this is a meaningful, if stylized, test of cache throughput.

The results, comparing this plugin’s performance with the Redis Cache by Till Krüss, are here. Lower numbers are faster. Times per operation are in microseconds.

It is remarkable how much faster this plugin fetches data from the cache than the Redis Cache does. That is probably because the php service processes must issue network requests and wait for responses while using Redis. When using SQLite, the php service process runs native sqlite.org code in the SQLite3 extension, which finds the cached item in a table. Because much of that table is likely already resident in the operating system’s RAM-cached file system, the retrieval operations are fast. This performance boost is consistent with the observations of SQLite’s developers.

Docket cache has comparable performance to SQLite Object Cache without APCu support, Lookup operations are significantly faster in SQLite Object Cache with APCu support.

atec APCu Cache is fastest. But it does not appear to be compatible with WP-CLI.

APCu Caching

Notice also that the APCu Cache and Object Cache 4 Everyone running in disk mode are extremely fast.

The APCu Cache is a php feature using similar technology to the opcode cache that speeds up the loading of php files. It is very fast, undoubtedly because it uses straightforward memory data structures and doesn’t require serialization. It is basically a persistent associative array.

It does have a limit on its number of slots — the maximum number of elements it can store. That limit is approximately 4096 elements, which is too small for an object cache on a site with more than a hundred or so posts or pages. It limits the amount of space as well, typically to 32M.

On Windows machines, the APCu cache is not shared between php worker processes, which futher limits its usefuless.

I wonder whether a three-level cache approach might work? The first level is RAM, the second would be APCu, and the third SQLite. This would need to be a write-through cache. Any writes would need to be placed both in APCu and SQLite.

Igbinary

This plugin also uses the Igbinary extension (when it is available) in place of php’s standard serialize() capability, because it generates a more compact storage format. The Redis cache uses serialize(). That accounts for some, but not all, the performance gains.

The test

The test is pretty cheesy as benchmarks go.

This WordPress shortcode handler performs the tests and emits how long they took.

function cachebenchmark( $atts, $content, $shortcode_tag ) {
	ob_start();
	$out   = array();
	$count = 10_000;
	$start = microtime( true );

	for ( $i = 0; $i < $count; $i ++ ) {
		$name = 'hackhack' . $i;
		set_transient( $name, [ $name, $i, str_pad( 'test', $i, 'x' ) ], 3600 );
	}
	$now   = microtime( true );
	$out[] = 'set_transient ' . number_format( ( $now - $start ) * 1_000_000 / $count, 0 ). 'μs';
	wp_cache_flush_runtime();
	$start = $now;
	for ( $i = 0; $i < $count; $i ++ ) {
		$name = 'hackhack' . $i;
		$data = get_transient( $name, 'missing' );
	}
	$now   = microtime( true );
	$out[] = 'get_transient ' . number_format( ( $now - $start ) * 1_000_000 / $count, 0 ). 'μs';
	wp_cache_flush_runtime();
	$start = $now;
	for ( $i = 0; $i < $count; $i ++ ) {
		$name = 'hackhack' . $i;
		delete_transient( $name, 'missing' );
	}
	$now   = microtime( true );
	$out[] = 'delete_transient ' . number_format( ( $now - $start ) * 1_000_000 / $count, 0 ). 'μs';
	$out[] = ob_get_clean();
	return implode( ' ', $out );
}

This ran on an Ubuntu 22.04 virtual machine wth 16GiB of memory and 6x 3.2GHz AMD Ryzen 7 cores, under a VirtualBox hypervisor on Windows 11. Versions: SQLite: 3.37.2 php: 8.3.16 Server: Apache/2.4.52 (Ubuntu) Plugin: 1.4.1 igbinary: available.

Use on a FreeBSD site

Something’s wrong on a site hosted on nearlyfreespeech.net. (They give each site its own FreeBSD jail.) Each SQLlite operation (lookup, save, etc) seems to take about 5 milliseconds. That’s obviously too long for the plugin to be useful.

Using gzdeflate on serialized objects

Using gzdeflate( igbinary_serialize ( $object )) seems to be extremely slow on large objects like the dashboard feeds. We probably shouldn’t do it.

Other Benchmarks Requested

If you have benchmark results, or advice on better tests, please share them in a comment here or on the plugin’s support forum.