Please allow the setting of a site speed sampling variable as per
"By default, a fixed 1% sampling of your site visitors make up the data pool from which the Site Speed metrics are derived. If you have a relatively small number of daily visitors to your site, such as 100,000 or fewer, you might want to adjust the sampling to a larger rate."
Smaller sites need to set the sample rate higher than the default 1% to get accurate data. Please enable this variable to be customized by user in the plugin panel.