Docs » Integrations Guide » Integrations Reference » Varnish

image0 Varnish

Metadata associated with the Varnish collectd plugin can be found here. The relevant code for the plugin can be found here.

DESCRIPTION

The Varnish collectd plugin collects metrics from Varnish and sends them to SignalFx.

Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture.

FEATURES

Built-in dashboards

  • Varnish (a): Overview of data from all Varnish servers.

image1

  • Varnish: Focus on a single Varnish server.

image2

REQUIREMENTS AND DEPENDENCIES

Software Version
collectd 5.1* or later (varnish 4.0+ requires collectd 5.5+)

INSTALLATION

  1. On RHEL/CentOS and Amazon Linux systems, run the following command to install this plugin:

    yum install collectd-varnish
    
  2. Download SignalFx’s sample configuration file to /etc/collectd/managed_config.

  3. Modify the sample configuration file to provide values that make sense for your environment, as described in Configuration below.

  4. Restart collectd.

Special Instructions for installing Varnish 3 on Ubuntu

Starting with collectd 5.6.1-sfx0, the varnish plugin provided within the SignalFx Package only supports varnish 4.
Follow these steps to add varnish 3 plugin support:
  1. Copy the varnish3.so file to the collectd library directory /usr/lib/collectd/.
  2. Update the configuration file 10-varnish.conf in your /etc/collectd/managed_config directory to show varnish3 instead of varnish as the name used by the LoadPlugin.
  3. Restart collectd.

Example configuration for Varnish 3 on Ubuntu

LoadPlugin varnish3
<Plugin "varnish">
  <Instance>
    CollectCache       true
    CollectConnections true
    CollectBackend     true
    CollectSHM         true
    CollectESI         true
    CollectFetch       true
    CollectHCB         true
    CollectSMA         true
    CollectSMS         true
    CollectSM          true
    CollectTotals      true
    CollectWorkers     true
    CollectUptime      true
    CollectVCL         true
    CollectStruct      true
    CollectObjects     true
    CollectSession     true
    CollectVSM         true
  </Instance>
</Plugin>

CONFIGURATION

There is no configuration required for the varnish collectd plugin. The default configuration is setup to bring in data on a large variety of metrics.

If using the sample configuration file 10-varnish.conf, no additional configuration is required. The following optional settings allow you to configure the metrics that will be sent using this plugin:

Setting Description Default
CollectBackend Back-end connection statistics, such as successful, reused, and closed connections. enabled
CollectCache Cache hits and misses. enabled
CollectConnections Number of client connections received, accepted and dropped. enabled
CollectESI Edge Side Includes (ESI) parse statistics. enabled
CollectFetch Statistics about fetches (HTTP requests sent to the backend). enabled
CollectHCB Inserts and look-ups in the crit bit tree based hash. Look-ups are divided into locked and unlocked look-ups. enabled
CollectSHM Statistics about the shared memory log, a memory region to store log messages which is flushed to disk when full. enabled
CollectSM file (memory mapped file) storage statistics. enabled
CollectSMA malloc or umem (umem_alloc(3MALLOC) based) storage statistics. The umem storage component is Solaris specific. enabled
CollectSMS synth (synthetic content) storage statistics. This storage component is used internally only. enabled
CollectTotals Collects overview counters, such as the number of sessions created, the number of requests and bytes transferred. enabled
CollectWorkers Collect statistics about worker threads. enabled

USAGE

Sample of built-in dashboard in SignalFx:

image3

METRICS

Field name Description Collected (Yes/No) Collectd option
uptime Child uptime No n/a
client_conn Client connections accepted Yes CollectConnections
client_drop Connection dropped, no sess Yes  
client_req Client requests received Yes  
cache_hit Cache hits Yes CollectCache
cache_hitpass Cache hits for pass Yes  
cache_miss Cache misses Yes  
backend_conn Backend conn. success Yes CollectBackend
backend_unhealthy Backend conn. not attempted Yes  
backend_busy Backend conn. too many Yes  
backend_fail Backend conn. failures Yes  
backend_reuse Backend conn. reuses Yes  
backend_toolate Backend conn. was closed Yes  
backend_recycle Backend conn. recycles Yes  
backend_unused Backend conn. unused Yes  
fetch_head Fetch head Yes CollectFetch
fetch_length Fetch with Length Yes  
fetch_chunked Fetch chunked Yes  
fetch_eof Fetch EOF Yes  
fetch_bad Fetch had bad headers Yes  
fetch_close Fetch wanted close Yes  
fetch_oldhttp Fetch pre HTTP/1.1 closed Yes  
fetch_zero Fetch zero len Yes  
fetch_failed Fetch failed Yes  
n_sess_mem N struct sess_mem No n/a
n_sess N struct sess No  
n_object N struct object No  
n_vampireobject N unresurrected objects No  
n_objectcore N struct objectcore No  
n_objecthead N struct objecthead No  
n_smf N struct smf No  
n_smf_frag N small free smf No  
n_smf_large N large free smf No  
n_vbe_conn N struct vbe_conn No  
n_wrk N worker threads Yes CollectWorkers
n_wrk_create N worker threads created Yes  
n_wrk_failed N worker threads not created Yes  
n_wrk_max N worker threads limited Yes  
n_wrk_queue N queued work requests Yes  
n_wrk_overflow N overflowed work requests Yes  
n_wrk_drop N dropped work requests Yes  
n_backend N backends No n/a
n_expired N expired objects No  
n_lru_nuked N LRU nuked objects No  
n_lru_saved N LRU saved objects No  
n_lru_moved N LRU moved objects No  
n_deathrow N objects on deathrow No  
losthdr HTTP header overflows No  
n_objsendfile Objects sent with sendfile No  
n_objwrite Objects sent with write No  
n_objoverflow Objects overflowing workspace No  
s_sess Total Sessions Yes CollectTotals
s_req Total Requests Yes  
s_pipe Total pipe Yes  
s_pass Total pass Yes  
s_fetch Total fetch Yes  
s_hdrbytes Total header bytes Yes  
s_bodybytes Total body bytes Yes  
sess_closed Session Closed No n/a
sess_pipeline Session Pipeline No  
sess_readahead Session Read Ahead No  
sess_linger Session Linger No  
sess_herd Session herd No  
shm_records SHM records Yes CollectSHM
shm_writes SHM writes Yes  
shm_flushes SHM flushes due to overflow Yes  
shm_cont SHM MTX contention Yes  
shm_cycles SHM cycles through buffer Yes  
sm_nreq allocator requests Yes CollectSM
sm_nobj outstanding allocations Yes  
sm_balloc bytes allocated Yes  
sm_bfree bytes free Yes  
sma_nreq SMA allocator requests Yes CollectSMA
sma_nobj SMA outstanding allocations Yes  
sma_nbytes SMA outstanding bytes Yes  
sma_balloc SMA bytes allocated Yes  
sma_bfree SMA bytes free Yes  
sms_nreq SMS allocator requests Yes CollectSMS
sms_nobj SMS outstanding allocations Yes  
sms_nbytes SMS outstanding bytes Yes  
sms_balloc SMS bytes allocated Yes  
sms_bfree SMS bytes freed Yes  
backend_req Backend requests made No n/a
n_vcl N vcl total No  
n_vcl_avail N vcl available No  
n_vcl_discard N vcl discarded No  
n_purge N total active purges No  
n_purge_add N new purges added No  
n_purge_retire N old purges deleted No  
n_purge_obj_test N objects tested No  
n_purge_re_test N regexps tested against No  
n_purge_dups N duplicate purges removed No  
hcb_nolock HCB Lookups without lock Yes CollectHCB
hcb_lock HCB Lookups with lock Yes  
hcb_insert HCB Inserts Yes  
esi_parse Objects ESI parsed (unlock) Yes CollectESI
esi_errors ESI parse errors (unlock) Yes  

Below is a list of all metrics.

Metric Name Brief Type
counter.varnish.cache-hit Count of cache hits. cumulative counter
counter.varnish.cache-hitpass Count of cache hits for pass cumulative counter
counter.varnish.cache-miss Count of misses cumulative counter
counter.varnish.connections-received The count of parseable client requests seen. cumulative counter
counter.varnish.connections-recycled Count of backend connection recycles cumulative counter
counter.varnish.connections-reuses Count of backend connection reuses cumulative counter
counter.varnish.session-accepted Count of sessions successfully accepted cumulative counter
counter.varnish.totals-req_body Total request body bytes received cumulative counter
counter.varnish.totals-req_header Total request header bytes received cumulative counter
counter.varnish.totals-resp_body Total response body bytes transmitted cumulative counter
counter.varnish.totals-resp_header Total response header bytes received cumulative counter
gauge.varnish.uptime Child varnish process uptime gauge

counter.varnish.cache-hit

cumulative counter

Count of cache hits

A cache hit indicates that an object has been delivered to a
client without fetching it from a backend server.

counter.varnish.cache-hitpass

cumulative counter

Count of hits for pass

A cache hit for pass indicates that Varnish is going to
pass the request to the backend and this decision has been
cached in it self. This counts how many times the cached
decision is being used.

counter.varnish.cache-miss

cumulative counter

Count of misses

A cache miss indicates the object was fetched from the
backend before delivering it to the backend.

counter.varnish.connections-received

cumulative counter

The count of parseable client requests seen

The count of parseable client requests seen by the server.

counter.varnish.connections-recycled

cumulative counter

Count of backend connection recycles

This counter is increased whenever we have a keep-alive
connection that is put back into the pool of connections.
It has not yet been used, but it might be, unless the backend
closes it.

counter.varnish.connections-reuses

cumulative counter

Count of backend connection reuses

This counter is increased whenever we reuse a recycled connection.

counter.varnish.session-accepted

cumulative counter

Count of sessions successfully accepted

counter.varnish.totals-req_body

cumulative counter

Total request body bytes received

Total bytes from request bodies.

counter.varnish.totals-req_header

cumulative counter

Total request header bytes received

Total bytes from request headers.

counter.varnish.totals-resp_body

cumulative counter

Total response body bytes transmitted

Total bytes from response bodies.

counter.varnish.totals-resp_header

cumulative counter

Total response header bytes received

Total bytes from response headers.

gauge.varnish.uptime

gauge

Child varnish process uptime

How long the child varnish process has been running.