Site speed measurement with curl

Maybe you can measure site speed performance with curl - Would be cool!

Purpose

Detect site speed decreases - As this is often an indication of impending doom - This is the main purpose
Measure absolute speed as a factor of hosting service level - Secondary reason.

Does it work?

My initial impression (2024.08) is mixed: The measurement times are way too small. I guess curl doesn't load all related objects, but only the plain html file
Curl doesn't parse HTML. It only downloads the indicated object - Usually a plain HTML file. If you want the whole page, wget might be a better tool.

How to measure with curl

To measure the speed of a website using curl in Bash, you can use the -w (write-out) option, which allows you to specify various metrics about the transfer. Example:

curl -o /dev/null -s -w "Time Total: %{time_total}\nTime Connect: %{time_connect}\nTime Pretransfer: %{time_pretransfer}\nTime Starttransfer: %{time_starttransfer}\n" <URL>

More elaborated

#!/bin/bash
#
url="example.com"

curl_arg="Timing metrics - ${url}"
#
curl_arg+="   Redirect: %{time_redirect}\n"
curl_arg+="   Name lookup: %{time_namelookup}\n"
curl_arg+="   Connect: %{time_connect}\n"
curl_arg+="   SSL/TLS handshake: %{time_connect}\n"
curl_arg+="   Pretransfer: %{time_pretransfer}\n"
curl_arg+="   Starttransfer: %{time_starttransfer}\n"
curl_arg+="   Total: %{time_total}\n"

curl -o /dev/null -s -w "${curl_arg}" $url

With

-w - write-out, to receive various details about the HTTP request and response. See placeholders below for details.
-s - silent, suppressing progress reporting and error messages
/dev/null - Discarding the output (the body of the response)

Include https://

Include https:// (or http:// if so desired) in the URL. Otherwise, you might get an empty response (and those are really fast!) or otherwise unprepresentative result.

Placeholders

Placeholders that can be used in combination with the -w write-out flag:

List of curl Placeholders
Category	Placeholder	Description
General Placeholders	%{url_effective}	Final URL after all redirections
	%{http_code}	HTTP response code
	%{http_connect}	HTTP response code of the CONNECT request
	%{response_code}	Same as %{http_code}
	%{redirect_count}	Number of redirections
	%{num_redirects}	Same as %{redirect_count}
	%{num_connects}	Number of connections made
Timing Metrics	%{time_total}	Total time taken for the transfer
	%{time_namelookup}	Time taken for DNS resolution
	%{time_connect}	Time taken to establish the connection
	%{time_appconnect}	Time taken to complete the SSL/TLS handshake
	%{time_pretransfer}	Time taken from the start until just before the transfer starts
	%{time_starttransfer}	Time taken from the start until the first byte is received
	%{time_redirect}	Time taken for redirections
Size Metrics	%{size_download}	Total size of the downloaded data
	%{size_upload}	Total size of the uploaded data
	%{size_header}	Total size of the headers
	%{size_request}	Total size of the request, including headers
	%{size_request_header}	Total size of the request headers
Speed Metrics	%{speed_download}	Average download speed
Speed Metrics	%{speed_upload}	Average upload speed
Other Metrics	%{content_type}	Content type of the response
	%{effective_url}	Final URL after redirection
	%{remote_ip}	Remote IP address of the server
	%{remote_port}	Remote port number
	%{local_ip}	Local IP address used for the connection
	%{local_port}	Local port number used for the connection
	%{protocol}	Protocol used for the transfer (e.g., HTTP/1.1, HTTP/2)

Sources

https://www.techrepublic.com/article/how-to-test-website-speeds-curl/ - Same approach as in the code above. It doesn't add anything
https://blog.josephscott.org/2011/10/14/timing-details-with-curl/ - Handy comments
https://github.com/mat/dotfiles/blob/master/bin/curlt
https://stackoverflow.com/questions/19344698/how-can-i-retrieve-google-page-speed-score-using-curl - Something different, but definately interesting

Site speed measurement with curl

Inhoud