August 9th: updated post to include data from July.
In a recent WordPress performance hangout, we talked about how we could assess the impact of the WordPress 6.2 release on production sites.
The data
One of the public datasets we discussed was the HTTP Archive’s Core Web Vitals Tech report, that queries monthly a number of sites based on popularity. It is able to detect WordPress sites. One of the metrics it reports is TTFB. Or, more specifically, the percentage of WordPress sites that have good TTFB (getting a response from the server in less than 0.8s):
By default, the HTTP Archive’s Core Web Vitals Tech report lists percent of “good CWV”. Select TTFB in the top-right icon that says “optional metrics” to see this graph.
WordPress 6.2 was released by the end of March 2023, so April would be the first month to include some sites using WordPress 6.2.
- April = 31.32%
- May = 33.16% (+1.84)
- June = 34.59% (+1.43)
- July = 35.16% (+0.57)
From April to July, there is almost a four-point increase. The ratio of improvement was 1.8 in May and 1.4 in June. It’s the first time since we have records that there have been two consecutive months with that kind of growth. More on this below.
Felix Arntz prepared a report based on alternative data, by Chrome: WordPress 6.2 performance impact in the field (CrUX). He was able to filter the dataset by WordPress version (6.2 vs. 6.1) and type of theme (classic vs. block). There’s more detail in there, so I recommend taking a look.
The TLDR is that this field report also confirms the lab benchmarks we ran for the release. TTFB for sites using WordPress 6.2 has improved compared to 6.1, specially when using block themes:
The caveat
The public real-world datasets are affected by changes everywhere: hosts updating the PHP version to a more performant one, browser’s updates, improvements in WordPress releases, plugins/themes getting faster, etc. There’s a lot to unpack.
We cannot isolate the impact a WordPress release has in the field like we do in the lab. However, the fact that the field data and the lab data report the same trends is a good thing. It connects the two. It suggests that improvements to the WordPress base are indeed influencing the real-world metrics in a way we can track.
The growth ratio of a WordPress release
In the recent versions, with the auto-updates, a new release surpasses the old one in a few days. The first weeks sites are updated en masse, and then it reaches a plateau. You can check this velocity by looking at the download counter and the stats. Right now, a few hours after the 6.3 release, WordPress 6.3 has 12M downloads while 6.2 sits at 100M (already at 12%).
This trend is the same we see for TTFB in the HTTP Archive report: higher growth in the month after the release (May, 1.8 points), and descending in the next ones (June with 1.4 points and July with 0.5 points, almost reaching a plateau).
This is what leads me to believe that, indeed, the WordPress base is driving the updates in performance we see in the public datasets.
What’s next?
Lab benchmarks report that 6.3 brings bigger performance improvements than 6.2. While 6.2 mostly improved TTFB, according to the analysis, 6.3 improves both TTFB and LCP significantly.
If the WordPress base is indeed driving the changes, we should see the public datasets impacted starting in August. Excited to see what the fall brings!
Leave a Reply