Percona Cookbook

Description

Installs the Percona MySQL client and/or
server components. (We are attempting to leverage the Sous-Chefs
MySQL cookbook as much as possible.)

Optionally installs:

XtraBackup hot backup software
Percona Toolkit advanced command-line tools
XtraDB Cluster high availability and high scalability solution for MySQL.

Maintainers

This cookbook is maintained by the Sous Chefs. The Sous Chefs are a community of Chef cookbook maintainers working together to maintain important cookbooks. If you’d like to know more please visit sous-chefs.org or come chat with us on the Chef Community Slack in #sous-chefs.

Requirements

Supported Platforms

We provide an expanding set of tests against the following 64-bit platforms which match what upstream supports:

CentOS 7+
Debian 10+
Ubuntu 18.04+ LTS

Cookbooks

Chef

This cookbook requires Chef >= 16.

Recipes

percona - The default which includes the client recipe.
percona::package_repo - Sets up the package repository and installs common packages.
percona::client - Installs the Percona MySQL client libraries.
percona::server - Installs and configures the Percona MySQL server daemon.
percona::backup - Installs and configures the Percona XtraBackup hot backup software.
percona::toolkit - Installs the Percona Toolkit software
percona::cluster - Installs the Percona XtraDB Cluster server components
percona::configure_server - Used internally to manage the server configuration.
percona::replication - Used internally to grant permissions for replication.
percona::access_grants - Used internally to grant permissions for recipes.
percona::ssl - Used internally to setup ssl certificates for server/client.

Resources

Usage

This cookbook installs the Percona MySQL components if not present, and pulls updates if they are installed on the
system.

This cookbook uses inclusion terminology where applicable replacing terms such as master/slave to source/replica
which matches the terminology decided upstream. Older
releases of Percona still use the terms in their configuration so those will remain, however we will be using the newer
terms with attributes, property and variable names. Currently both terms should work however the next major release of
this cookbook will only use the new terminology.

Encrypted Passwords

This cookbook requires Encrypted Data Bags. If you forget to use them or do not use a node attribute to overwrite them empty passwords will be used.

To use encrypted passwords, you must create an encrypted data bag. This cookbook assumes a data bag named passwords, but you can override the name using the node['percona']['encrypted_data_bag'] attribute. You can also optionally specify a data bag secret file to be loaded for the secret key using the node['percona']['encrypted_data_bag_secret_file'] attribute.

This cookbook expects a mysql item and a system item. Please refer to the official documentation on how to get this setup. It actually uses a MySQL example so it can be mostly copied. Ensure you cover the data bag items as described below.

You also may set expected item names via attributes node['percona']['encrypted_data_bag_item_mysql'] and node['percona']['encrypted_data_bag_item_system'].

Skip passwords

Set the ['percona']['skip_passwords'] attribute to skip setting up passwords. Removes the need for the encrypted data bag if using chef-solo. Is useful for setting up development and ci environments where you just want to use the root user with no password. If you are doing this you may want to set ['percona']['server']['debian_username'] to be "root" also.

Skip Configure

Set the ['percona']['skip_configure'] attribute to skip having the server recipe include the configure_server recipe directly after install. This is mostly useful in a wrapper cookbook sort of context. Once skipped, you can then perform any pre-config actions your wrapper needs to, such as dropping a custom configuration file or init script or cleaning up incorrectly sized innodb logfiles. You can then include configure_server where necessary.

mysql item

The mysql item should contain entries for root, backup, and replication. If no value is found, the cookbook will fall back to the default non-encrypted password.

system item

The "system" item should contain an entry for the debian system user as specified in the node['percona']['server']['debian_username'] attribute. If no such entry is found, the cookbook will fall back to the default non-encrypted password.

Example: "passwords" data bag - this example assumes that node['percona']['server']['debian_username'] = spud

{
  "mysql" :
  {
    "root" : "trywgFA6R70NO28PNhMpGhEvKBZuxouemnbnAUQsUyo=\n"
    "backup" : "eqoiudfj098389fjadfkadf=\n"
    "replication" : "qwo0fj0213fm9020fm2023fjsld=\n"
  },
  "system" :
  {
    "spud" : "dwoifm2340f024jfadgfu243hf2=\n"
  }
}

Above shows the encrypted password in the data bag. Check out the encrypted_data_bag_secret setting in knife.rb to setup your data bag secret during bootstrapping.

Install client development package

To install the package including header files needed to compile software using the client library (percona-server-devel on Centos and libperconaserverclient-dev on Debian), set node['percona']['client']['install_devel_package'] to true. This will add those packages to the list to be installed when running the percona::client recipe. This attribute is disabled by default.

Replication over SSL

To enable SSL based replication, you will need to flip the attribute node['percona']['server']['replication']['ssl_enabled'] to true and add a new data_bag item
to the percona encrypted data_bag (seenode['percona']['encrypted_data_bag'] attribute) with the id ssl_replication ( see node['percona']['encrypted_data_bag_item_ssl_replication'] attribute) that contains this data:

{
  "id": "ssl_replication",
  "ca-cert": "CA_CERTIFICATE_STRING",
  "server": {
    "server-cert": "SERVER_CERTIFICATE_STRING",
    "server-key": "SERVER_KEY_STRING"
  },
  "client": {
    "client-cert": "CLIENT_CERTIFICATE_STRING",
    "client-key": "CLIENT_KEY_STRING"
  }
}

All certificates and keys have to be converted to a string (easiest way is to use ruby: /usr/bin/env ruby -e 'p ARGF.read' <filename>) and placed
instead of CA_CERTIFICATE_STRING, SERVER_CERTIFICATE_STRING, SERVER_KEY_STRING, CLIENT_CERTIFICATE_STRING, CLIENT_KEY_STRING.

Percona XtraDB Cluster

Below is a minimal example setup to bootstrap a Percona XtraDB Cluster. Please see the official documentation for more information. This is not a perfect example. It is just a sample to get you started.

Wrapper recipe recipes/percona.rb:

# Setup the Percona XtraDB Cluster
cluster_ips = []
unless Chef::Config[:solo]
  search(:node, 'role:percona').each do |other_node|
    next if other_node['private_ipaddress'] == node['private_ipaddress']
    Chef::Log.info "Found Percona XtraDB cluster peer: #{other_node['private_ipaddress']}"
    cluster_ips << other_node['private_ipaddress']
  end
end

cluster_ips.each do |ip|
  firewall_rule "allow Percona group communication to peer #{ip}" do
    source ip
    port 4567
    action :allow
  end

  firewall_rule "allow Percona state transfer to peer #{ip}" do
    source ip
    port 4444
    action :allow
  end

  firewall_rule "allow Percona incremental state transfer to peer #{ip}" do
    source ip
    port 4568
    action :allow
  end
end

cluster_address = "gcomm://#{cluster_ips.join(',')}"
Chef::Log.info "Using Percona XtraDB cluster address of: #{cluster_address}"
node.override['percona']['cluster']['wsrep_cluster_address'] = cluster_address
node.override['percona']['cluster']['wsrep_node_name'] = node['hostname']

include_recipe 'percona::cluster'
include_recipe 'percona::backup'
include_recipe 'percona::toolkit'

Example percona role roles/percona.rb:

name "percona"
description "Percona XtraDB Cluster"

run_list 'recipe[paydici::percona]'

default_attributes(
  "percona" => {
    "server" => {
      "role" => "cluster"
    },

    "cluster" => {
      "package"                     => "percona-xtradb-cluster-56",
      "wsrep_cluster_name"          => "percona_cluster_1",
      "wsrep_sst_receive_interface" => "eth1" # can be eth0, public, private, etc.
    }
  }
)

Now you need to bring three servers up one at a time with the percona role applied to them. By default the servers will sync up via rsync server state transfer (SST)

Explicit my.cnf templating

In some situations it is preferable to explicitly define the attributes needed in a my.cnf file. This is enabled by adding categories to the node['percona']['conf'] attributes. All keys found in the node['percona']['conf'] map will represent categories in the my.cnf file. Each category contains a map of attributes that will be written to the my.cnf file for that category. See the example for more details.

Example

node['percona']['conf']['mysqld']['slow_query_log_file'] = "/var/lib/mysql/data/mysql-slow.log"

This configuration would write the mysqld category to the my.cnf file and have an attribute slow_query_log_file whose value would be /var/lib/mysql/data/mysql-slow.log.

Example output (my.cnf)

[mysqld]
slow_query_log_file = /var/lib/mysql/data/mysql-slow.log

Dynamically setting the bind address

There's a special attribute node['percona']['server']['bind_to'] that allows you to dynamically set the bind address. This attribute accepts the values "public_ip", "private_ip", "loopback", or and interface name like "eth0". Based on this, the recipe will find a corresponding ipv4 address, and override the node['percona']['server']['bind_address'] attribute.

Goals

In no particular order:

Be the most flexible way to setup a MySQL distribution through Chef
- Support for Chef Solo
- Support for Chef Server
Support the following common database infrastructures:
- Single server instance
- Traditional Source/Replica replication
- Multi-source cluster replication
Support the most recent Chef runtime environments
Be the easiest way to setup a MySQL distribution through Chef

TODO

Fully support all of the standard Chef-supported distributions

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Added some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

Contributors

This project exists thanks to all the people who contribute.

Backers

Thank you to all our backers!

Dependent cookbooks

yum >= 0.0.0

yum-epel >= 0.0.0

line >= 0.0.0

Contingent cookbooks

percona Cookbook CHANGELOG

This file is used to list changes made in each version of the percona cookbook.

3.4.8 - 2024-05-23

Standardise files with files in sous-chefs/repo-management

3.4.7 - 2024-05-06

3.4.6 - 2024-05-06

3.4.5 - 2023-12-21

3.4.4 - 2023-09-28

3.4.3 - 2023-09-04

3.4.2 - 2023-07-10

3.4.1 - 2023-06-14

Fix adjustment of open files limit within systemd unit

3.4.0 - 2023-06-07

Avoid changing the perms on /tmp
Remove duplicate configuration line
Avoid duplicating mysqld configuration section in my.cnf
Remove the open file limit from the systemd unit since it is already set in my.cnf anyway

3.3.1 - 2023-05-17

3.3.0 - 2023-04-25

Fixed the user key in the ctrl_hash for the run_query method in the mysql_user custom resource
Updated ctrl_hash (now user_hash) parameters to validate if a user can login using username@host instead of username@ctrl_host
Wrapped password substring in single quotes to protect from special characters in sql_command_string helper method
Updated default encoding and collate in 8.0 to utf8mb4 and utf8mb4_0900_ai_ci respectively
Fixed 8.0 client rpm dependency conflict on centos 7
Stop running tests for Server and Client 5.6
Change test behaviour to not enable jemalloc on RHEL 9.

3.2.11 - 2023-04-25

Standardise files with files in sous-chefs/repo-management

3.2.10 - 2023-04-07

Standardise files with files in sous-chefs/repo-management

3.2.9 - 2023-04-01

Standardise files with files in sous-chefs/repo-management

3.2.8 - 2023-04-01

Standardise files with files in sous-chefs/repo-management

3.2.7 - 2023-04-01

Standardise files with files in sous-chefs/repo-management

3.2.6 - 2023-03-20

Standardise files with files in sous-chefs/repo-management

3.2.5 - 2023-03-15

Standardise files with files in sous-chefs/repo-management

3.2.4 - 2023-02-23

Standardise files with files in sous-chefs/repo-management

3.2.3 - 2023-02-14

Standardise files with files in sous-chefs/repo-management

3.2.2 - 2023-02-14

Standardise files with files in sous-chefs/repo-management

3.2.1 - 2022-12-08

Standardise files with files in sous-chefs/repo-management

3.2.0 - 2022-08-18

Remove myisam_repair_threads for < 5.7 as it's been deprecated
Remove delivery and move to calling RSpec directly via a reusable workflow
Update tested platforms
Add support to Alma Linux & Rocky Linux
Update default encoding when using 8.0 to utf8mb3 to fix idempotency issues
Fix management of server my.cnf
Set manage_symlink_source to false and set force_unlink to true to fix idempotency on Debian based systems
Fix issue when testing on Debian dokken images by removing mailutils (and thus mysql-common)

3.1.3 - 2022-02-17

Standardise files with files in sous-chefs/repo-management

3.1.2 - 2022-02-08

Remove delivery folder

3.1.1 - 2022-01-03

Fix root password setting for version >= 5.7

3.1.0 - 2021-10-04

Use dnf_module resource from yum cookbook instead of manually disabling module

3.0.0 - 2021-09-17

Chef 17 updates: enable unified_mode on all resources
Remove dependency on openssl cookbook and create percona_secure_random method to replace that functionality
Remove dependency on chef-vault cookbook and require Chef >= 16
Use full gpg key id for apt repos and switch to using keyserver.ubuntu.com
Move mysql dnf module disabling to before yum repos

2.1.2 - 2021-08-30

Standardise files with files in sous-chefs/repo-management

2.1.1 - 2021-06-01

Standardise files with files in sous-chefs/repo-management

2.1.0 - 2021-01-19

Fix error when granting multi-word privileges (ex. REPLICATION CLIENT) to users through percona_mysql_user
Remove support for Ubuntu 16.04

2.0.1 - 2020-12-15

Fix links to resources in README

2.0.0 - 2020-10-23

Add percona_mysql_database resource for creating, querying and removing databases
Add percona_mysql_user resource for creating, modifying and removing database users
Remove mysql_chef_gem and mysql2_chef_gem providers in favor of the percona_mysql_database and percona_mysql_user resources

1.1.1 - 2020-09-16

resolved cookstyle error: libraries/helpers.rb:125:1 refactor: ChefCorrectness/IncorrectLibraryInjection

1.1.0 - 2020-08-20

Add devel package attribute to client recipe

1.0.0 - 2020-08-14

Add support for Debian 10 for 5.7 only
Add support for Ubuntu 20.04 for 5.7 only
Add support for CentOS 8
Add cluster suite and tests to test cluster recipe
Add support for Percona 8.0 and default to that version
Re-add ChefSpec tests
Add ssl suite and tests for ssl recipe
Add master suite and tests for testing the replication recipe
Suite to test compatibility with Chef 13
@2.0.1
Update apt gpg key
Fixes for supporting 5.7
Fix manage_symlink_source warning with template[/etc/mysql/my.cnf]
Don't remove mysql-libs on RHEL
jemalloc package installation and path setup for all platforms
Fixed enabled ChefSpec tests
Use the correct syntax on 8.0 for SSL replication
Use correct cert path for master/slave
Fix issue when trying to set node['percona']['version'] in a recipe
Convert to InSpec tests and refactor test cookbook recipes
Don't install abi_version packages on Debian/Ubuntu
Standardise files with files in sous-chefs/repo-management
Move client package installation for cluster to cluster recipe
Remove support for Amazon Linux
Remove support for OpenSUSE
Remove support for Debian 8 (EOL)
Remove support for Fedora / Scientific
Remove support for CentOS 6
Remove references to EOL 5.5 release

Deprecated

Deprecate monitoring recipe
Use new inclusive terminology and add deprecation warning for old terms

0.17.2 - 2020-08-06

@2.0.1

Fix debian_password as a string for testing
Fix idempotency issues with find_password method

0.17.1 - 2020-05-14

resolved cookstyle error: recipes/access_grants.rb:28:40 convention: Layout/TrailingWhitespace
resolved cookstyle error: recipes/access_grants.rb:28:41 refactor: ChefModernize/FoodcriticComments
resolved cookstyle error: recipes/configure_server.rb:102:25 convention: Layout/TrailingWhitespace
resolved cookstyle error: recipes/configure_server.rb:102:26 refactor: ChefModernize/FoodcriticComments
resolved cookstyle error: recipes/configure_server.rb:170:42 convention: Layout/TrailingWhitespace
resolved cookstyle error: recipes/configure_server.rb:170:43 refactor: ChefModernize/FoodcriticComments
resolved cookstyle error: recipes/replication.rb:28:35 convention: Layout/TrailingWhitespace
resolved cookstyle error: recipes/replication.rb:28:36 refactor: ChefModernize/FoodcriticComments

0.17.0 - 2020-05-05

resolved cookstyle error: attributes/default.rb:8:16 warning: Lint/SendWithMixinArgument
resolved cookstyle error: libraries/passwords.rb:23:16 refactor: ChefModernize/DatabagHelpers
resolved cookstyle error: recipes/ssl.rb:17:9 refactor: ChefModernize/DatabagHelpers
Removed unused use_inline_resources and whyrun_supported? methods from the resources
Removed unused long_description metadata from metadata.rb
Simplify platform checks inn only_if checks
Remove the unused .foodcritic file
Update metadata to point to Sous Chefs
Migrate to github actions for testing

[0.16.5]

Use latest percona GPG keys for yum repo. See New Percona Package Signing Key Requires Update on RHEL and CentOS

0.16.1 - 2015-06-03

Many changes

Collaborator Number Metric

3.4.8 passed this metric

Contributing File Metric

3.4.8 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file

Cookstyle Metric

3.4.8 passed this metric

No Binaries Metric

3.4.8 passed this metric

Testing File Metric

3.4.8 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file

Version Tag Metric

3.4.8 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number

Adoptable Cookbooks List

Supermarket Belongs to the Community

View our collection of guides, documentation, and articles

Select Badges

Select Supported Platforms

Select Status

Percona Cookbook

Description

Maintainers

Requirements

Supported Platforms

Cookbooks

Chef

Recipes

Resources

Usage

Encrypted Passwords

Skip passwords

Skip Configure

mysql item

system item

Install client development package

Replication over SSL

Percona XtraDB Cluster

Explicit my.cnf templating

Example

Example output (my.cnf)

Dynamically setting the bind address

Goals

TODO

Contributing

Contributors

Backers

Sponsors

Dependent cookbooks

Contingent cookbooks

percona Cookbook CHANGELOG

3.4.8 - 2024-05-23

3.4.7 - 2024-05-06

3.4.6 - 2024-05-06

3.4.5 - 2023-12-21

3.4.4 - 2023-09-28

3.4.3 - 2023-09-04

3.4.2 - 2023-07-10

3.4.1 - 2023-06-14

3.4.0 - 2023-06-07

3.3.1 - 2023-05-17

3.3.0 - 2023-04-25

3.2.11 - 2023-04-25

3.2.10 - 2023-04-07

3.2.9 - 2023-04-01

3.2.8 - 2023-04-01

3.2.7 - 2023-04-01

3.2.6 - 2023-03-20

3.2.5 - 2023-03-15

3.2.4 - 2023-02-23

3.2.3 - 2023-02-14

3.2.2 - 2023-02-14

3.2.1 - 2022-12-08

3.2.0 - 2022-08-18

3.1.3 - 2022-02-17

3.1.2 - 2022-02-08

3.1.1 - 2022-01-03

3.1.0 - 2021-10-04

3.0.0 - 2021-09-17

2.1.2 - 2021-08-30

2.1.1 - 2021-06-01

2.1.0 - 2021-01-19

2.0.1 - 2020-12-15

2.0.0 - 2020-10-23

1.1.1 - 2020-09-16

1.1.0 - 2020-08-20

1.0.0 - 2020-08-14

Deprecated

0.17.2 - 2020-08-06

0.17.1 - 2020-05-14

0.17.0 - 2020-05-05

[0.16.5]

0.16.1 - 2015-06-03

Collaborator Number Metric 3.4.8 passed this metric

Collaborator Number Metric

3.4.8 passed this metric

Contributing File Metric

3.4.8 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file

Cookstyle Metric

3.4.8 passed this metric

No Binaries Metric

3.4.8 passed this metric

Testing File Metric

3.4.8 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file

Version Tag Metric

3.4.8 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number