This directory contains a collection of Chef Infra cookbooks and recipes to provision+configure the code-dot-org repository, Ruby applications and various OS components and services to servers.

The top-level cookbook/recipe is cdo-apps::default , which includes most all other recipes used to provision+configure the applications.

These cookbooks are used to provision applications to production and other managed-infrastructure server environments. In these managed environments, the application's CloudFormation template provides its EC2 instances with a cloud-init user-data script that includes the bootstrap_chef_stack.sh.erb script-template, which installs the Chef Client and invokes the chef-client executable which runs the specified cookbooks/recipes.

Currently, managed environments run update_cookbook_versions and package/upload versioned cookbooks to a hosted Chef Infra Server as part of their CI build, then connect to that server with chef-client to provision.

adhoc environments work slightly differently by running chef-client in Local Mode against locally-stored cookbooks. This is much simpler and will eventually become the default provisioning mode for all environments.

In the future, Chef cookbooks may also be used to provision applications to Docker images/layers and/or local-development workstations.

We aim to follow best practices for test driven infrastructure with Chef, though tools and techniques are rapidly evolving, and overall coverage is still far from complete. Currently, we try to cover basic functionality with integration tests for any new development or changes to existing cookbooks, so that we can test the behavior of a cookbook/recipe in isolation without having to provision the entire application stack.

Chef integration tests run using Test Kitchen, are located in the test/ subdirectory in each cookbook, and are configured by .kitchen.yml files.

To test a cookbook, run bundle install to install test-kitchen and dependencies, cd to the cookbook directory, then run:

• bundle exec kitchen create to create the platform environment (e.g., Docker container, EC2 instance)
• bundle exec kitchen converge to install Chef and converge the cookbook in the platform environment
• bundle exec kitchen verify to run the integration test suite
• bundle exec kitchen destroy to destroy the created platform environment and release associated resources

Tests run by default on Docker containers using the kitchen-docker driver. First make sure Docker is installed and running locally.

Tests can be configured to run on live Amazon EC2 instances using the kitchen-ec2 driver.

To use the EC2 configuration instead of Docker, run (from within the cookbook directory):

KITCHEN_LOCAL_YAML=../.kitchen.ec2.yml bundle exec kitchen [create|converge|verify|destroy]

Note that the live EC2 instance will continue consuming resources until destroy is called, so make sure to do this at the end of your testing!

When adding a new dependency to an individual cookbook, there are three places you need to declare the dependency:

1. The cookbook-specific Berksfile; ie, cookbook 'pyenv' in cookbooks/cdo-python/Berksfile
2. The cookbook-specific metadata.rb; ie, depends 'pyenv' in cookbooks/cdo-python/metadata.rb
3. The top-level Berksfile; ie, cookbook 'pyenv' in cookbooks/Berksfile

Note that adhocs will not automatically pick up updates to cookbook code.

When we render the template that defines the bootstrap script, it uploads the output of berks package to S3 as a side effect. That artifact is then downloaded to /var/chef/cookbooks as part of initial bootstrapping, but will not thereafter be automatically updated.