Announcing Coherence 2.0 and CNC, the first open source IaC framework
All posts

GCP with Terraform: Managing Infrastructure

Learn how to manage your infrastructure on Google Cloud Platform (GCP) with Terraform. Explore prerequisites, initial setup, Terraform configuration, state management, modules, scaling, and best practices.

Zan Faruqui
September 18, 2024

Managing your infrastructure on Google Cloud Platform (GCP) with Terraform allows you to automate the setup and maintenance of your tech resources, such as servers and networks, using code. Here's what you need to know to get started:

  • Prerequisites: Have a GCP account, a GCP project, Terraform installed, and a basic understanding of GCP.

  • Initial Setup: Create a GCP project, enable necessary services, create a Terraform service account, and set up authentication.

  • Terraform Configuration: Write Terraform scripts using HCL to define your infrastructure, such as VM instances and networks.

  • State Management: Store Terraform state in a remote backend like Google Cloud Storage for team collaboration and safety.

  • Modules: Utilize Terraform modules to reuse and organize infrastructure components.

  • Scaling: Manage resources at scale with multiple workspaces and module graphs.

  • Best Practices: Treat infrastructure configuration as code, automate testing and deployments, and document everything.

This approach offers consistency, efficiency, collaboration, safety, flexibility, cost savings, and better management of your GCP infrastructure.

Prerequisites

Before diving into Terraform on GCP, make sure you have:

  • An account with GCP

  • A GCP project ready to go

  • Terraform installed on your computer

  • A basic understanding of how GCP works

Initial Setup of GCP Environment

Here's how to get your GCP ready for Terraform:

  • If you don't have a GCP project yet, create one

  • Turn on the GCP services you'll need, like the Compute Engine and Cloud Resource Manager

  • Make a special account for Terraform to use

  • Download a key for this account and save it as a JSON file

  • Use gcloud auth application-default login to let Terraform use your GCP credentials

Now, your GCP is all set for Terraform.

Writing Your First Terraform Configuration

Terraform uses a language called HCL to write down what you want it to do. Here's a simple example that sets up a computer server (VM instance) and a network (VPC) on GCP:

Configure GCP provider

provider "google" { credentials = file("terraform-svc-account.json") project = "" region = "us-central1" }

Create VPC network

resource "google_compute_network" "default" { name = "terraform-network" }

Create Compute Engine VM instance

resource "google_compute_instance" "default" { name = "terraform-instance" machine_type = "e2-medium"

boot_disk { initialize_params { image = "debian-cloud/debian-9" } }

network_interface { network = google_compute_network.default.name access_config { } } }


Before you make any changes, check your Terraform files to make sure everything looks right. This step helps catch mistakes.

## Managing Terraform [State](https://www.terraform.io/docs/state/) in GCP

![State](https://mars-images.imgix.net/seobot/screenshots/www.terraform.io-47330efe8785fd1e27611e050e9e7ca7.jpg?auto=compress)

When you use Terraform to set up your cloud stuff on GCP, it keeps track of everything it does in a file called the Terraform state. By default, this file is kept on your computer, which can be tricky when you're working in a team. Moving this file to a shared place, like Google Cloud Storage (GCS), makes it easier for everyone to work together without stepping on each other's toes.

### Benefits of [Remote State](https://www.terraform.io/docs/state/remote.html) Storage

![Remote State](https://mars-images.imgix.net/seobot/screenshots/www.terraform.io-13139163ebbd5b9daf606f4d2f05d055.jpg?auto=compress)

Putting your Terraform state file in a GCS bucket is smart because:

- It stops you from losing your [state](https://www.terraform.io/docs/state/) file if something goes wrong with your computer.
- It locks the state file so that only one person can make changes at a time, keeping things safe.
- It lets everyone in your team see and use the state file, making teamwork smoother.
- It works well with other GCP tools, making your project more connected.

Using GCS for your Terraform state means everyone can work together more easily and safely.

### Configuring GCS Backend for Terraform State

Here's how to set up a GCS bucket for your Terraform state:

- Make a GCS bucket in your GCP project

```bash
PROJECT_ID=$(gcloud config get-value project)
gsutil mb gs://${PROJECT_ID}-tfstate
  • Turn on versioning for the bucket
gsutil versioning set on gs://${PROJECT_ID}-tfstate
  • Change your backend.tf file to use the bucket
terraform {
  backend "gcs" {
    bucket = "PROJECT_ID-tfstate"
    prefix = "env/dev" 
  }
}
  • Put your actual project ID where it says PROJECT_ID

  • Save your changes and upload them to your code repository

For more info, check out the Terraform backend documentation.

By using a GCS bucket, you make it easier for your team to use Terraform on GCP. It helps everyone stay on the same page and makes your project work better with other cloud tools.

Reusing Infrastructure Elements with Terraform Modules

Terraform modules let you group parts of your cloud setup, like networks or servers, into neat packages. This way, you don't have to write out every single detail every time you want to set something up in Google Cloud Platform (GCP). Think of modules as a way to keep your Terraform scripts tidy and easy to use.

Benefits of Modular Infrastructure

Here are some good things about using modules:

  • Simplified configurations: Putting stuff like networks into modules keeps your main script simple.

  • Reusability: Write your code once in a module, and use it many times. This saves you from writing the same thing over and over.

  • Encapsulation: Your main script doesn't get messy with details. Modules show only what you need to know.

  • Organization: Keeping things in modules helps you stay organized, especially as your project gets bigger.

  • Sharing: It's easy to share modules with others in your team or company.

  • Customization: You can change how a module works by giving it different inputs.

Using Public Terraform Modules for GCP

Terraform Modules

You can find ready-to-use modules for GCP on the Terraform Registry or from Google's Cloud Foundation Toolkit. These modules are made by experts and help you set up GCP services quickly without having to figure out all the details yourself.

Creating Custom Terraform Modules

You can also make your own modules for things you use a lot, like a set of rules for your network. Here's how to do it:

  • Write a Terraform script for what you want, like network.tf.

  • Add inputs so you can change how it works for different projects.

  • Make sure to output any info that other parts of your setup might need.

  • Put your script in a modules directory.

  • Use this module in your main Terraform script to set it up.

This way, you can easily use your network setup in different projects by just calling your module. Over time, you'll build up a collection of modules that make setting up new projects faster and easier.

Managing Resources at Scale with Terraform

Leveraging Multiple Workspaces

Workspaces in Terraform help you handle different setups like development, testing, and production separately. This way, you can avoid mixing them up and making unintended changes.

Here's why workspaces are handy:

  • They keep each environment's setup apart with their own state files.

  • You can use the same setup (like modules and providers) across different environments.

  • Different teams can work on their own environments without stepping on each other's toes.

  • You build your modules once and use them everywhere, which saves time.

To get started with workspaces:

  1. Pick a place to keep your state files safe and sound, like Google Cloud Storage.

  2. Set it up to handle multiple workspaces.

    terraform {
      backend "gcs" { 
        bucket = "tfstate-bucket"
        
        prefix = "env"
      }
    }
    
  3. Make a new workspace for each environment you have.

    $ terraform workspace new dev
    $ terraform workspace new test
    $ terraform workspace new prod
    
  4. Choose the workspace you want to use.

    $ terraform workspace select dev
    

This setup lets you manage big projects easily by keeping each environment's setup separate.

Composing Infrastructure with Module Graphs

When your setup gets big and complicated, understanding how everything connects is key. Terraform uses something called a module graph to show how different parts depend on each other.

Here are some tips for putting your setup together well:

  • Break your setup into smaller parts (modules).

  • Clearly show how these parts connect using the graph.

  • Keep the complicated stuff inside the modules.

  • Only show what you need to outside of the modules.

  • Put modules together to make up your whole environment.

  • Use the same modules in different places to save effort.

For instance, you could have a module for managing a network, which you then use in another module for setting up a Kubernetes cluster. This cluster module could then be part of a bigger setup that includes other things.

This way of doing things lets you change parts without messing up the rest. It's all about keeping things simple and organized.

In short, using graphs and modules helps you handle big and complex setups by organizing them into smaller, manageable pieces.

sbb-itb-550d1e1

Infrastructure as Code Best Practices with Terraform

Treat Infrastructure Configuration as Code

Think of your infrastructure setup like any other code you write. This means you should:

  • Use Git to keep track of your infrastructure files. This helps you see changes over time, go back if something goes wrong, and try out new ideas safely.

  • Make sure to review changes in your setup files, just like you would with regular code. This helps catch mistakes.

  • Use tools to make sure your code looks clean and follows the same style. This makes it easier for everyone to understand.

  • Test your setup to catch problems early. This keeps your infrastructure running smoothly.

These steps help make sure your infrastructure is reliable and easy to manage.

Automate Testing and Deployments

Make your life easier by automating how you test and deploy your infrastructure with Terraform. This means:

  • Automatically checking your setup to make sure it follows the rules.

  • Testing your infrastructure automatically to find issues before they become big problems.

  • Setting up your resources without having to do everything by hand.

This speeds things up, reduces mistakes from doing things manually, and makes sure everything is checked regularly.

Document Infrastructure Design and Configurations

Keep a clear record of your infrastructure, including:

  • Drawings that show how everything is set up.

  • Explanations of what each part does and why it's important.

  • Details about how things are configured and how they work together.

This helps everyone understand the setup, makes it easier to fix things when needed, and keeps important knowledge from getting lost.

Conclusion

When you use Terraform with GCP to handle your tech setup using code, you get a bunch of perks:

Consistency and Reliability

Writing down your tech setup in code means you can make sure everything's set up the same way every time. This stops small differences from sneaking in.

Efficiency and Agility

With Terraform, you can do things faster because the computer does the heavy lifting. This makes it easy to set up or remove things as needed.

Collaboration and Sharing

Keeping Terraform's setup info in one place online makes it easier for teams to work together. And using the same pieces of code in different projects saves time.

Safety and Governance

Using good practices, like keeping track of changes and checking your work, makes things safer and lets you keep a close eye on everything.

Flexibility and Portability

Terraform works with lots of different services, so you're not stuck with just one. And using bits of code that do specific things means you can change things around without starting from scratch.

Cost Savings

Using code to manage your tech stuff means you can automate tasks to use resources better and save money over time.

By treating your tech setup as code and using Terraform's tools, businesses can work better, more reliably, and come up with new ideas on Google Cloud Platform. The Infrastructure Manager service adds extra help and connects with other tools for an even smoother process.

How do you manage infrastructure using Terraform?

Terraform lets you write down how you want your computer servers and networks to be set up using its own special language. You can keep these instructions in a version control system like Git. This is how Terraform makes managing stuff easier:

  • Infrastructure as Code (IaC) - Your setup instructions are the main guide. Any changes are checked and approved.

  • Execution Plans - Terraform shows you what it will do before it does anything, so you can check first.

  • State Management - Terraform keeps track of what it has set up and how everything connects, and you can keep this info in a safe place like Cloud Storage.

  • Automated Testing - You can set up tests for your Terraform setup to find problems early on.

  • Modularity - You can organize your setup into smaller, reusable parts to make it easier to handle.

How do you create infrastructure in GCP using Terraform?

Here's a simple guide:

  • Get a GCP project ready

  • Install Terraform

  • Write a Terraform config file for GCP

  • Set up GCP provider and credentials

  • Get Terraform ready to go

  • Check and tidy up your config

  • Look over the plan Terraform suggests

  • Apply the config to set up your resources in GCP

This way, you can use Terraform to automatically set up your GCP resources.

How Terraform cloud helps to manage infrastructure?

Terraform Cloud helps by providing a central place to manage your setups. It offers features like safe storage for your setup info, ways to control who can do what, rules to follow, notifications, and estimates on costs. This makes it easier for teams to work together on their setups.

Does Terraform work with GCP?

Yes, Terraform works well with GCP. It has a special GCP provider that lets you set up things like computer servers, Kubernetes clusters, networks, and storage. Terraform makes it easier to use code to set up and manage your GCP resources.

Related posts