cookbook 'mercury', '= 1.5.18'
mercury
(59) Versions
1.5.18
-
-
1.5.35
-
1.5.34
-
1.5.33
-
1.5.32
-
1.5.31
-
1.5.30
-
1.5.29
-
1.5.28
-
1.5.27
-
1.5.26
-
1.5.25
-
1.5.24
-
1.5.23
-
1.5.22
-
1.5.21
-
1.5.20
-
1.5.19
-
1.5.18
-
1.5.17
-
1.5.16
-
1.5.15
-
1.5.14
-
1.5.13
-
1.5.12
-
1.5.11
-
1.5.10
-
1.5.9
-
1.5.8
-
1.5.7
-
1.5.6
-
1.5.5
-
1.5.4
-
1.5.3
-
1.5.1
-
1.5.0
-
1.4.4
-
1.4.3
-
1.4.2
-
1.4.1
-
1.4.0
-
1.3.9
-
1.3.8
-
1.3.7
-
1.3.6
-
1.3.5
-
1.3.4
-
1.3.3
-
1.3.2
-
1.3.1
-
1.3.0
-
1.2.9
-
1.2.8
-
1.2.7
-
1.2.6
-
1.2.4
-
1.2.3
-
1.2.2
-
1.2.1
-
1.2.0
Follow1
- 1.5.35
- 1.5.34
- 1.5.33
- 1.5.32
- 1.5.31
- 1.5.30
- 1.5.29
- 1.5.28
- 1.5.27
- 1.5.26
- 1.5.25
- 1.5.24
- 1.5.23
- 1.5.22
- 1.5.21
- 1.5.20
- 1.5.19
- 1.5.18
- 1.5.17
- 1.5.16
- 1.5.15
- 1.5.14
- 1.5.13
- 1.5.12
- 1.5.11
- 1.5.10
- 1.5.9
- 1.5.8
- 1.5.7
- 1.5.6
- 1.5.5
- 1.5.4
- 1.5.3
- 1.5.1
- 1.5.0
- 1.4.4
- 1.4.3
- 1.4.2
- 1.4.1
- 1.4.0
- 1.3.9
- 1.3.8
- 1.3.7
- 1.3.6
- 1.3.5
- 1.3.4
- 1.3.3
- 1.3.2
- 1.3.1
- 1.3.0
- 1.2.9
- 1.2.8
- 1.2.7
- 1.2.6
- 1.2.4
- 1.2.3
- 1.2.2
- 1.2.1
- 1.2.0
Installs/Configures mercury
cookbook 'mercury', '= 1.5.18', :supermarket
knife supermarket install mercury
knife supermarket download mercury
Opscode Cookbook that Installs and configures Mercury Global Loadbalancer.
What is Mercury?
Mercury is a Global loadbalancer, designed to add a dns based loadbalancing layer on top of its internal loadbalancer or 3rd pary loadbalancers such as cloud services
This makes mercury able to loadbalance across multiple cloud environments using dns, while keeping existing cloud loadbancer sollutions in place
- Source: https://github.com/schubergphilis/mercury
- Binaries: https://github.com/schubergphilis/mercury/releases
- Docs: http://mercury-global-loadbalancer.readthedocs.io/en/latest/
Requirements
- Chef 12.5+
Platforms
- RHEL 6+, CentOS6+
- RHEL 7+, CentOS7+
- RHEL 8+, CentOS8+
Documentation
Install
Include the default recipe
Configuration items
default['mercury']['settings'] = {
manage_network_interfaces: "(yes|no)" -- allow mercury to add vip's to the network interfaces (default: yes) - required for internal proxy or for haproxy who does not add vip's
enable_proxy: "(yes|no)" -- use internal proxy for loadbalancing (default: yes) - not needed for external proxy programs, or dns only setup
}
default['mercury']['logging'] = {
level: "(debug|info|warn|error)" -- log level
output: "(stdout|file)" -- log output
}
default['mercury']['cluster'] = {
name: "" -- cluster group name
binding: "myhost" -- ip/interface to bind on, also acts as cluster name
settings: {
connection_timeout: 10, -- connection timeout for cluster nodes
connection_retry_count: 10, -- connection retry for cluster nodes
connection_retry_interval: 10, -- connection retry interval for cluster nodes
ping_interval: 10, -- ping interval for cluster nodes
ping_timeout: 10, -- time to wait for reply before discarding cluster node
port: 9000, -- port for cluster communications
tls: { -- see tls settings below for more details
insecureskipverify: true
}
},
nodes: [
{
search: "recipe:chef_recipe", -- let chef find the remote nodes based on search
port: 80 -- connect to port 80 of this node
},
]
}
default['mercury']['dns'] = {
binding: 'myhost' -- ip to bind on
port: 53 -- port to listen on for dns queries
allowed_requests: [ "A", "AAAA" ] -- what records to respond to (default is to allow most requests)
}
default['mercury']['web'] = {
binding: 'myhost' -- ip to bind on
port: 9001 -- port to listen to for web interface
tls: { -- see tls settings below
}
auth: {
password: {
"username": "sha256hashOfPassword"
}
ldap: {
host: 'ldaphost' -- ldap host to connect to
port: 389 -- ldap port to connect to, 389 for tls, 636 for ssl
method: 'tls' -- method of connection, tls or ssl
binddn: "OU=Users,DC=example,DC=com" -- search path to find users
filter: "(&(objectClass=organizationalPerson)(uid=%s))" -- filter path to find user
domain: 'example' -- domain to prepend to ldap login
}
}
}
default['mercury']['loadbalancer']['pools'] = {
'poolname' => { -- name of the pool/vip
listener: {
ip: "" -- ip of the vip
port: 0 -- port of the vip
mode: "(udp|tcp|http|https)" -- what protocol to listen to
tls: { -- see mercury src/tlsconfig/tls.go for all available options, not supplying this will use defaults
minversion: 'VersionTLS12', -- minimum version of TLS to allow
ciphersuites: %w(TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384), -- ciphersuite to allow
curvepreferences: %w(CurveP521) -- curve preference to use
}
readtimeout: 0 -- time to wait for http request from client (0 = unlimited, 10 = default)
writetimeout: 0 -- time to wait for server reply towards client (0 = unlimited, 0 = default)
ocspstapling: true|false -- enables ocsp stapling (true = default)
},
outboundacls: [
{
action: '(add|remove|replace)', -- what to do with header
header_key: 'Location', -- header key to do action with
header_value: 'https://###REQ_HOST######REQ_PATH###' -- header value to set - not required on remove
},
{
action: '(add|remove|replace)', -- what to do with cookie
cookie_key: 'stky', -- cookie key to do action with
cookie_value: '###NODE_ID###', -- calculated variable or string
cookie_expire: '24h', -- time for cookie (ex. 1m, 3s, 24h)
cookie_secure: true, -- secure
cookie_httponly: true -- httponly
},
],
inboundacls: [
-- see outboundacls, but actions can also include allow/deny on inbound acls
{
action: '(allow|deny)', -- what to do with cidr
cidrs: ["127.0.0.1/8", "10.0.0.0/8"] -- network cidrs to apply action to
},
{
action: '(add|remove|replace)', -- what to do with header
header_key: 'Location', -- header key to do action with
header_value: 'https://###REQ_HOST######REQ_PATH###' -- header value to set - not required on remove
},
{
action: '(allow|deny)', -- what to do with header (regex are allowed to allow/deny rules)
header_key: 'User-Agent', -- header key to do action with
header_value: ".*Macintash.*" -- value regex match for allow/denies are allowed
},
],
errorpage: {
file: "/var/mercury/sorry.html" -- specifying a sorry page, enables showing this on errors
trigger_threshold: 500 -- http result code when to show sorry page (e.g. 500 will show the sorrypage on all errors of 500+)
},
healthchecks: [{ // >= mercury version 0.10 - allows an array of healthchecks on a Pool
type: '(httpget|httppost|tcpconnect|icmpping|udpping|tcpping)',
ip: '1.2.3.4',
pingpackets: 4,
pingtimeout: 1
}],
backends: {
'backendname' => {
inboundacls: [] -- see earlier inbound acls
outboundacls: [] -- see earlier inbound acls
hostnames: [""] -- hostnames to repply on with this backend
dnsentry: {
hostname: "www" -- hostname to give dns record for
domain: "domain.org" -- domain where this host subsides
ip: "" -- IPv4 of the vip to supply on dns record (will use listener ip above if this is not specified
ip6: "" -- IPv6 of the vip to supply on dns record (will use listener ip above if this is not specified
},
healthcheck: { // <= mercury version 0.9.x - allows only a single healthcheck
type: '(httpget|httppost|tcpconnect)',
httpstatus: 200,
httpheaders: ['Content-Type: application/soap+xml; charset=utf-8'],
postdata: '',
request: "http://www.domain.org/",
tls: {
insecureskipverify: true
}
},
healthcheckmode: "(any|all)" -- default: "all" - should 1 or all checks be ok for backend to be online
healthchecks: [{ // >= mercury version 0.10 - allows an array of healthchecks
type: '(httpget|httppost|tcpconnect)',
httpstatus: 200,
httpheaders: ['Content-Type: application/soap+xml; charset=utf-8'],
postdata: '',
request: "http://www.domain.org/",
tls: {
insecureskipverify: true
}
}],
balance: {
method: 'topology,leastconnected' -- see loadbalancing methods for more details
local_topology: "site" -- see networks below
preference: 0123 -- priority for preference based loadbalancing (lower has higher preference)
active_passive: "(yes|no)" -- default: no - affects monitoring: to only alert if 0 or >1 nodes are online
clusternodes: int -- default: #cluster_nodes - affects monitoring: usefull for vips that only live on 1 cluster node
}
nodes: [
{
search: "recipe:chef_recipe", -- let chef find the remote nodes based on search
port: 80 -- connect to port 80 of this node
},
{
hostname: 'hostname' -- specify host (optional)
ip: '1.2.3.4' -- specify ip (required)
port: 80 -- connect to port 80 of this node
}
],
connectmode: "(udp|tcp|http|https)", -- what protocol to connect to the backend with
}
}
}
}
default['mercury']['loadbalancer']['networks'] = { -- used for topology loadbalancing
'site' => {
cidrs = [ "127.0.0.1/32" ]
}
}
default['mercury']['dns']['domains']['domain.org'] = { -- local dns records, also uses TTL as default for loadbalanced records
'ttl' => 10, -- default ttl for all records
'soa' => { -- standard soa record
name: 'mydomain.org',
ns: "ns.mydomain.org",
email: "hostmaster.mydomain.org",
refresh: 30, # time that a slave will refresh from master
retry: 30, # retry time if slave failed to connect to master
expire: 3600, # cache on slave
minimum: 10, # minimum cache time for slave
},
'records' => [ -- records that are not loadbalanced but local
{
name: "" -- for domain records no name needed
target: "10 mymx.domain.org" -- mx server to send to
type: "MX" -- MX record type
},
{
name: "www2" -- hostname in domain record
target: "::1" -- target
type: "AAAA" -- ipv6 record
}
]
}
default['mercury']['dns']['allow_forwarding'] = [ "10.10.0.177/32", "::1/128" ] -- allows recursive dns queries for given clients (used when setting mercury as dns forwarding server)
Loadbalancing Methods
leastconnected - based on current clients connected
leasttraffic -- based on traffic generated
preference -- based on preference set in node of backend
responsetime -- based on responsetime of a backend node (first byte response? or http roundtrip?)
random -- rng
roundrobin -- try to switch them a bit
-
sticky -- based on the 'stky' cooky which contains the node, requires the following 2 acls:
{ action: 'replace', cookie_key: 'stky', cookie_value: '###NODE_ID###', cookie_expire: '24h', cookie_secure: true, cookie_httponly: true, cookie_path: '/' }, { action: 'add', cookie_key: 'stky', cookie_value: '###NODE_ID###', cookie_expire: '24h', cookie_secure: true, cookie_httponly: true, cookie_path: '/' },
note that for stickyness in combination with global loadbalancing, you need to
add each node to all loadbalancers
. so that the lb can always forward the client to its previous node topology -- based on topology network set, requires the network to be set
firstavailable -- used for compatibility reasons, this ensures that we always return only the first record, and prevents multiple hosts in dns responses.
Loadbalancing Performance
When selecting multiple loadbalance options, keep in mind the speed of each of them.
See below a benchmark of the speeds.
```
$ go test -bench=.
BenchmarkBalancerLeastConnected-4 10000000 160 ns/op
BenchmarkBalancerLeastTraffic-4 10000000 167 ns/op
BenchmarkBalancerPreference-4 10000000 163 ns/op
BenchmarkBalancerRandom-4 200000 9643 ns/op
BenchmarkBalancerRoundRobin-4 5000000 165 ns/op
BenchmarkBalancerSticky-4 10000000 124 ns/op
BenchmarkBalancerTopology-4 2000000 837 ns/op
BenchmarkBalancerResponseTime-4 10000000 136 ns/op
```
TLS Settings
TLS settings should be set on the Mercury Cluster and Webservices. But they can be set on Listeners and Backend.
Setting TLS settings only on backends and not on a listener means that SNI will be used to identify a site for the given SSL certificate.
tls: {
minversion: minimal tls version to allow
ciphersuites: cipersuites to allo
curvepreferences: curve preference to use
insecureskipverify: allow a client to connect to insecure SSL certificate (hostname mismatches)
certificatefile: certificate crt file to use - this should be a path to a file
certificatekey: certificate key file to use - this should be a path to a file
databagname: name of the databag to find the SSL certificate
databagitem: item name of the databag to find the SSL certificate
}
TLS Min version
The minimum TLS version to allow.
Options:
* VersionSSL30,
* VersionTLS10,
* VersionTLS11,
* VersionTLS12,
TLS Cypher suites
The allowed Cypers, see https://golang.org/pkg/crypto/tls/#pkg-constants for details
Options:
* TLS_RSA_WITH_RC4_128_SHA
* TLS_RSA_WITH_3DES_EDE_CBC_SHA
* TLS_RSA_WITH_AES_128_CBC_SHA
* TLS_RSA_WITH_AES_256_CBC_SHA
* TLS_RSA_WITH_AES_128_CBC_SHA256
* TLS_RSA_WITH_AES_128_GCM_SHA256
* TLS_RSA_WITH_AES_256_GCM_SHA384
* TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
* TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
* TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
* TLS_ECDHE_RSA_WITH_RC4_128_SHA
* TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
* TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
* TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
* TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
* TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
* TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
* TLS_FALLBACK_SCSV
TLS Recommended Cyphers and HTTP/2:
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 <- has to be first if you want HTTP/2 support!
The 4 cipers below are need for the best SSL-Labs certificate but do not support HTTP/2, the HTTP/2 one will slightly downgrade your score
* TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* TLS_RSA_WITH_AES_256_GCM_SHA384
* TLS_RSA_WITH_AES_256_CBC_SHA
TLS Curve preferences
The tls curve preferences, see https://golang.org/pkg/crypto/tls/#pkg-constants for details
Options:
* CurveP256
* CurveP384
* CurveP521
* X25519
SSL Certificates
You can use databags to load the certificate, or let them be generated.
Automaticly generated certificates
If you supply only the file names on where to find the certificates, the cookbook will generate these certificates and used them in Mercury
Example:
tls: {
certificatefile: "my_certificate.crt"
certificatekey: "my_certificate.key"
}
This wil generate the certificates in the OS default's SSL certificate path, and genrate certificates with the name my_certificate.crt + my_certificate.key
Databag controlled certificates
If you supply a databagname, the cookbook will search the databag for the keys provided as file name in the databagitem
Example:
tls: {
databagname: 'mercury'
databagitem: 'certifactes_production'
certificatefile: "my_certificate_file"
certificatekey: "my_certificate_key"
}
This will get the certificate from the data bag called 'mercury' with the item 'certificates_production', and search for the keys 'my_certificate_file' and 'my_certificate_key'
The data bag will look something like this:
```
$ knife data bag show mercury certificates_aroductionp --secret-file secrey_key -Fj
{
"id": "certificates_production",
"my_certificate_file": "-----BEGIN CERTIFICATE-----\nMII...\n..aNZ\n-----END CERTIFICATE-----"
}
```
Examples
Below You can find some example configurations
HTTPS VIP example
Simple site, that serves a website using SSL offloading - we allow SSL connects, and use HTTP connects to the backend
default['mercury']['loadbalancer']['pools']['my_backend_https'] = {
listener: { ip: 'myapp_vip', port: 443, mode: 'https' },
backends: {
'myapp_backend' => {
hostnames: ["myapp.mydomain.org"],
dnsentry: { hostname: 'myapp', domain: 'mydomain.org' },
healthcheck: { type: 'httpget', httpstatus: 200, request: "http://myapp.mydomain.org/" },
balance: { method: 'sticky,roundrobin' },
nodes: [{ search: "recipe:myapp.mydomain.org AND chef_environment:#{node.chef_environment}", port: 80 }],
connectmode: 'http'
}
}
}
This will accept connections on 'myapp_vip', which will do a chef search for the nodes serving this application
HTTP -> HTTPS redirect example
all requests on this ip on port 80 are redirected to their original domain and path to https://
default['mercury']['loadbalancer']['pools']['http_redirect'] = {
listener: { ip: matrix_ip['backend'], port: 80, mode: 'http' },
outboundacls: [
{ action: 'add', header_key: 'Location', header_value: 'https://###REQ_HOST######REQ_PATH###' },
{ action: 'add', status_code: 301 }
],
backends: {
'redirect' => {
hostnames: ['default'],
connectmode: 'internal'
}
}
}
HTTP redirect only as default to main website
only requests to myapp.mydomain.org are served, all other requests are redirected to https://myapp.mydomain.org
also we only allow private networks to connect to this site
default['mercury']['loadbalancer']['pools']['my_backend_https'] = {
listener: { ip: 'myapp_vip', port: 443, mode: 'https' },
backends: {
'myapp_backend' => {
hostnames: ["myapp.mydomain.org"],
dnsentry: { hostname: 'myapp', domain: 'mydomain.org' },
healthcheck: { type: 'httpget', httpstatus: 200, request: "http://myapp.mydomain.org/" },
balance: { method: 'sticky,roundrobin' },
nodes: [{ search: "recipe:myapp.mydomain.org AND chef_environment:#{node.chef_environment}", port: 80 }],
connectmode: 'http'
},
'redirect' => {
outboundacls: [
{ action: 'add', header_key: 'Location', header_value: 'https://myapp.mydomain.org' },
{ action: 'add', status_code: 301 }
],
inboundacls: [
{ action: 'allow', cidrs: ["127.0.0.1/8", "10.0.0.0/8"] }
],
hostnames: ['default'],
connectmode: 'internal'
}
}
}
HTTP serve site on all domains
all requests to this ip/port serve this website
default['mercury']['loadbalancer']['pools']['my_backend_https'] = {
listener: { ip: 'myapp_vip', port: 443, mode: 'https' },
backends: {
'myapp_backend' => {
hostnames: ["default"],
dnsentry: { hostname: 'myapp', domain: 'mydomain.org' },
healthcheck: { type: 'httpget', httpstatus: 200, request: "http://myapp.mydomain.org/" },
balance: { method: 'sticky,roundrobin' },
nodes: [{ search: "recipe:myapp.mydomain.org AND chef_environment:#{node.chef_environment}", port: 80 }],
connectmode: 'http'
}
}
}
Reload and Config updates
A reload should be enough to apply config changes. but some changes will affect clients.
Only when changing the Listener variables of a config, or the TLS config of backends, requires the proxy part of the laodbalancer for this specific Pool to be reloaded.
As such, when changing the IP, port, Connection mode or TLS config, clients currently connected to this pool will be disconnected, and can reconnect once the pool is up again. (mater of miliseconds)
All other configuration items are changable without interruption to the client.
Error/Sorrypage handling
When a error page is set in the config, it will always show on internal errors (no backend available, or acl allow/deny)
For errors given by a webserver you can use the trigger_threshold
which will only trigger errors if the status code is equal or higher.
If you do not want the sorry page to show on return codes from the webserver, then set this to a higher number then the http error codes (e.g. 600 or up)
ACLS
ACL's can be set to add/replace/modify headers, or to allow/deny requests based on headers/cidr (see examples above)
to use ALLOW/DENY, you must use the INBOUND
acl. you cannot mix allow and deny ACL's together, this will result in only the allow beeing processed.
License & Authors
- Author:: Ronald Doorn (rdoorn@schubergphilis.com)
Copyright:: Schuberg Philis Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Dependent cookbooks
ssl_certificate ~> 2.1.0 |
logrotate ~> 2.1 |
Contingent cookbooks
There are no cookbooks that are contingent upon this one.
CHANGE LOG for Mercury
This file is used to list changes made in each major version of Mercury.
1.2.6 till Current
Changes:
* Updated to latest version of mercury
1.2.5
Changes:
* Now defaults to Mercury version 0.12.3
1.2.3
Changes:
* Now defaults to Mercury version 0.12.1
1.2.2
Changes:
* Now defaults to Mercury version 0.12.0
1.2.1
Changes:
* Now defaults to Mercury version 0.11.1
1.2.0
Changes:
* Now supports Mercury version 0.10.0 and up
Important:
With support of multiple healthchecks the healthcheck attribute has been renamed to healthchecks and this is now an array.
In terms of chef attributes that means that you can convert: healthcheck: {} to healthchecks: [{}].
This has been made backwards compatible so that using the old healthcheck will keep working, but do NOT combine both healthcheck and healthchecks as only 1 will work.
1.1.0:
Changes:
* Now supports Mercury version 0.9.0 and up
* Adjust default log level to info
1.0.0:
- Start of change log
Collaborator Number Metric
1.5.18 failed this metric
Failure: Cookbook has 1 collaborators. A cookbook must have at least 2 collaborators to pass this metric.
Contributing File Metric
1.5.18 passed this metric
Foodcritic Metric
1.5.18 passed this metric
No Binaries Metric
1.5.18 failed this metric
Failure: Cookbook should not contain binaries. Found:
mercury/logo.png
Testing File Metric
1.5.18 passed this metric
Version Tag Metric
1.5.18 passed this metric
1.5.18 failed this metric
1.5.18 passed this metric
Foodcritic Metric
1.5.18 passed this metric
No Binaries Metric
1.5.18 failed this metric
Failure: Cookbook should not contain binaries. Found:
mercury/logo.png
Testing File Metric
1.5.18 passed this metric
Version Tag Metric
1.5.18 passed this metric
1.5.18 passed this metric
1.5.18 failed this metric
Failure: Cookbook should not contain binaries. Found:
mercury/logo.png
Testing File Metric
1.5.18 passed this metric
Version Tag Metric
1.5.18 passed this metric
1.5.18 passed this metric
1.5.18 passed this metric