You are probably familiar with the awesome Prometheus Blackbox Exporter. The exporter works great for monitoring availability of ICMP, HTTP(S), TCP and DNS targets.

That said, when monitoring websites, I have found that it would be very helpful to provide insight as to why a probe failed, if possible.

The Blackbox exporter will happily report whether a probe was successful or not, but besides that there is no information available about the failure. This is usually fine when Prometheus best practices are concerned - after all, Prometheus is designed for whitebox monitoring, so this assumes that the endpoint is available.

Meet the Curl HTTP Exporter

I have made an effort to write a rather basic exporter wrapped around Curl in Python. The benefit of using Curl is access to many built-in reports, such as CURLINFO_OS_ERRNO and the ability to fetch an error code on failure.

This means, that upon failure, the exporter will report a metric with the error that has occurred.

This is, in my opinion, very helpful when troubleshooting the why part of things.

The exporter and instructions can be found in the Github repository.

In essence, once you start the exporter, you can use it more or less the same way the as the Blackbox Exporter (sans the module parameter):

- job_name: curlhttp_exporter
    scrape_timeout: 15s
    scrape_interval: 20s
    metrics_path: /
    static_configs:
    - targets:
      - https://www.mysite1.com/
      - https://ihaveasecondsite.com/
    relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: IP_ADDR:10080 # exporter location

As a bonus, you also get access to Curl settings via the Configuration:

  connecttimeout: 10         # connect timeout seconds
  timeout: 10                # second timeout for curl functions
  ipresolve: ipresolve_v4    # use ipv4? this maps to CURL_IPRESOLVE_V4
  useragent: Mozilla/5.0 ... # custom user-agent

SSL Information

The Blackbox Exporter is helpful with its information about the certificate expiry. I've taken this a step forward and, via CURLINFO_CERTINFO, the exporter provides information about the SSL certificates involved in the request. For example:

curlhttp_certinfo{target="https://expired.badssl.com/",subject_c="GB",subject_st="Greater Manchester",subject_l="Salford",subject_o="COMODO CA Limited",subject_cn="COMODO RSA Certification Authority",issuer_c="GB",issuer_st="Greater Manchester",issuer_l="Salford",issuer_o="COMODO CA Limited",issuer_cn="COMODO RSA Certification Authority",version="2",start_date="2010-01-19 00:00:00 GMT",expire_date="2038-01-18 23:59:59 GMT",signature="0a:.......:74:"} 1
curlhttp_certinfo_expire_date{target="https://expired.badssl.com/",signature="0a:.......:74:"} 2147468399
curlhttp_certinfo_start_date{target="https://expired.badssl.com/",signature="0a::.......:74:"} 1263855600

The certificate metrics can be joined by the signature label, which is unique to each SSL certificate.

If you plan on monitoring website(s) from multiple locations, I would suggest using SSL information on only one of them, to avoid feeding too much data into Prometheus (the metric labels are quite long).