Skip to content

Commit 957cf6d

Browse files
author
jtimberman
committed
Add database cookbook
1 parent 829fb02 commit 957cf6d

15 files changed

Lines changed: 708 additions & 0 deletions

‎database/README.txt‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
Database Cookbook
2+
=================
3+
4+
Configures mysql database masters and slaves and uses EBS for storage, integrating together with the application cookbook utilizing data bags for application related information.
5+
6+
This cookbook is written primarily to use MySQL and the Opscode mysql cookbook. Other RDBMS may be supported at a later date.
7+
8+
This cookbook does not automatically restore database dumps, but does install tools to help with that.
9+
10+
---
11+
Recipes
12+
=======
13+
14+
ebs_volume
15+
----------
16+
17+
Loads the aws information from the data bag. Searches the applications data bag for the database master or slave role and checks that role is applied to the node. Loads the EBS information and the master information from data bags. Uses the aws cookbook LWRP, `aws_ebs_volume` to manage the volume.
18+
19+
On a master node:
20+
* if we have an ebs volume already as stored in a data bag, attach it.
21+
* if we don't have the ebs information then create a new one and attach it.
22+
* store the volume information in a data bag via a ruby block.
23+
24+
On a slave node:
25+
* use the master volume information to generate a snapshot.
26+
* create the new volume from the snapshot and attach it.
27+
28+
Also on a master node, generate some configuration for running a snapshot via `chef-solo` from cron.
29+
30+
On a new filesystem volume, create as XFS, then mount it in /mnt, and also bind-mount it to the mysql data directory (default /var/lib/mysql).
31+
32+
master
33+
------
34+
35+
Loads the AWS information from a data bag. Create the databases for the application per the role checking magic, if they don't exist. Install s3cmd and create a config file so we can do manual database restores from dumpfiles if necessary. Finally store the replication status in a data bag.
36+
37+
slave
38+
-----
39+
40+
Retrieve the master status from a data bag, then start replication using a ruby block.
41+
42+
snapshot
43+
--------
44+
45+
Run via chef-solo. Retrieves the db snapshot configuration from the specified JSON file. Uses the `mysql_database` LWRP to lock and unlock tables, and does a filesystem freeze and EBS snapshot.
46+
47+
---
48+
Deprecated Recipes
49+
==================
50+
51+
The following recipes are no longer used as they have been deprecated in functionality both the above.
52+
53+
ebs_backup
54+
----------
55+
56+
Older style of doing mysql snapshot and replication using Adam Jacob's [ec2_mysql][0] script and library
57+
58+
[0]: http://github.com/adamhjk/ec2_mysql
59+
60+
---
61+
Data Bags
62+
=========
63+
64+
This recipe uses the apps data bag item for the specified application; see the `application` cookbook's README.txt. It also creates data bag items in a bag named 'aws' for storing volume information. In order to interact with EC2, it expects aws to have a main item:
65+
66+
{
67+
"id": "main",
68+
"ec2_private_key": "private key as a string",
69+
"ec2_cert": "certificate as a string",
70+
"aws_account_id": "",
71+
"aws_secret_access_key": "",
72+
"aws_access_key_id": ""
73+
}
74+
75+
Note: with the Open Source Chef Server, the server using the database recipes must be an admin client or it will not be able to create data bag items. You can modify whether the client is admin by editing it with knife.
76+
77+
knife client edit <client_name>
78+
{
79+
...
80+
"admin": true
81+
...
82+
}
83+
84+
This is not required if the Chef Server is the Opscode Platform.
85+
86+
---
87+
Usage
88+
=====
89+
90+
Aside from the application data bag (see the README.txt in the application cookbook), create a role for the database master in the chef-repo:
91+
92+
% vi roles/my_app_database_maser.rb
93+
name "my_app_database_master"
94+
description "Set up a MySQL DB master for the my_app application"
95+
run_list(
96+
"recipe[my_app_database]",
97+
"recipe[database::ebs_volume]",
98+
"recipe[database::master]"
99+
)
100+
% knife role from file roles/my_app_database_master.rb
101+
102+
The cookbook `my_app_database` is recommended to set up any application specific database resources such as configuration templates, trending monitors, etc. It is not required, but you would need to create it separately in `site-cookbooks`.
103+
104+
License and Author
105+
==================
106+
107+
Author:: Adam Jacob (<adam@opscode.com>)
108+
Author:: Joshua Timberman (<joshua@opscode.com>)
109+
Author:: AJ Christensen (<aj@opscode.com>)
110+
111+
Copyright 2009-2010, Opscode, Inc.
112+
113+
Licensed under the Apache License, Version 2.0 (the "License");
114+
you may not use this file except in compliance with the License.
115+
You may obtain a copy of the License at
116+
117+
http://www.apache.org/licenses/LICENSE-2.0
118+
119+
Unless required by applicable law or agreed to in writing, software
120+
distributed under the License is distributed on an "AS IS" BASIS,
121+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
122+
See the License for the specific language governing permissions and
123+
limitations under the License.

‎database/metadata.rb‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
maintainer "Opscode, Inc."
2+
maintainer_email "cookbooks@opscode.com"
3+
license "Apache 2.0"
4+
description "Sets up the database master or slave"
5+
long_description IO.read(File.join(File.dirname(__FILE__), 'README.txt'))
6+
version "0.5"
7+
8+
%w{ mysql aws xfs }.each do |cb|
9+
depends cb
10+
end

‎database/recipes/default.rb‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#
2+
# Author:: Joshua Timberman (<joshua@opscode.com>)
3+
# Cookbook Name:: database
4+
# Recipe:: default
5+
#
6+
# Copyright 2009, Opscode, Inc.
7+
#
8+
# Licensed under the Apache License, Version 2.0 (the "License");
9+
# you may not use this file except in compliance with the License.
10+
# You may obtain a copy of the License at
11+
#
12+
# http://www.apache.org/licenses/LICENSE-2.0
13+
#
14+
# Unless required by applicable law or agreed to in writing, software
15+
# distributed under the License is distributed on an "AS IS" BASIS,
16+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
17+
# See the License for the specific language governing permissions and
18+
# limitations under the License.
19+
#
20+

‎database/recipes/ebs_backup.rb‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
#
2+
# Author:: Joshua Timberman (<joshua@opscode.com>)
3+
# Cookbook Name:: database
4+
# Recipe:: ebs_backup
5+
#
6+
# Copyright 2009, Opscode, Inc.
7+
#
8+
# Licensed under the Apache License, Version 2.0 (the "License");
9+
# you may not use this file except in compliance with the License.
10+
# You may obtain a copy of the License at
11+
#
12+
# http://www.apache.org/licenses/LICENSE-2.0
13+
#
14+
# Unless required by applicable law or agreed to in writing, software
15+
# distributed under the License is distributed on an "AS IS" BASIS,
16+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
17+
# See the License for the specific language governing permissions and
18+
# limitations under the License.
19+
#
20+
21+
begin
22+
aws = Chef::DataBagItem.load(:aws, :main)
23+
Chef::Log.info("Loaded AWS information from DataBagItem aws[#{aws['id']}]")
24+
rescue
25+
Chef::Log.fatal("Could not find the 'main' item in the 'aws' data bag")
26+
raise
27+
end
28+
29+
db_role = String.new
30+
db_master_role = String.new
31+
db_type = node[:database][:type]
32+
33+
search(:apps) do |app|
34+
db_role = app["database_#{db_type}_role"] & node.run_list.roles
35+
db_master_role = app["database_master_role"]
36+
end
37+
38+
ebs_info = Chef::DataBagItem.load(:aws, "ebs_#{db_master_role}_#{node[:app_environment]}")
39+
40+
gem_package "dbi"
41+
gem_package "dbd-mysql"
42+
43+
directory "/mnt/aws-config" do
44+
mode 0700
45+
owner "root"
46+
group "root"
47+
end
48+
49+
template "/mnt/aws-config/config" do
50+
source "aws_config.erb"
51+
variables(
52+
:access_key => aws['aws_access_key_id'],
53+
:secret_key => aws['aws_secret_access_key']
54+
)
55+
owner "root"
56+
group "root"
57+
mode 0600
58+
end
59+
60+
git "/opt/ec2_mysql" do
61+
repository "git://github.com/jtimberman/ec2_mysql.git"
62+
reference "HEAD"
63+
action :sync
64+
not_if { ::FileTest.directory?("/opt/ec2_mysql/.git") }
65+
end
66+
67+
%w{backup restore}.each do |file|
68+
template "/usr/local/bin/db-#{file}.sh" do
69+
source "ebs-db-#{file}.sh.erb"
70+
owner "root"
71+
group "root"
72+
mode 0700
73+
variables(
74+
:mysql_root_passwd => node[:mysql][:server_root_password],
75+
:mysql_device => node[:mysql][:ebs_vol_dev],
76+
:ebs_vol_id => ebs_info['volume_id']
77+
)
78+
end
79+
end
80+
81+
if db_type == "master" && node[:app_environment] == "production"
82+
template "/etc/cron.d/db-backup" do
83+
source "ebs-backup-cron.erb"
84+
owner "root"
85+
group "root"
86+
mode 0644
87+
backup false
88+
end
89+
end

0 commit comments

Comments
 (0)