↓Skip to main content
  1. Posts/

[K3s-01] How to install K3s and setup Terraform

·476 words
K3s
Table of Contents

Terraform Example #

According to official K3s requirements, there are some prerequisites for setting up K3s nodes. For every nodes, following steps are required:

  1. ufw disable
  2. Be aware of VXLAN port “8472”

The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disables access to port 8472.

According to system requirements, instance size micro in AWS is possible. But, instance size small is recommended, based on my personal experience.

variable "vpc_id" { default = "vpc-xxx" }
variable "vpc_cidr" { default = "172.31.0.0/16" }
variable "subnet_id" { default = "subnet-xxx" }

resource "aws_instance" "k3s" {
  count = 3
  # ubuntu 22.04
  ami           = "ami-086cae3329a3f7d75"
  instance_type = "t2.small"
  key_name      = aws_key_pair.k3s_key.key_name

  # associate_public_ip_address = false

  network_interface {
    device_index         = 0
    network_interface_id = aws_network_interface.k3s_network_interface[count.index].id
  }

  tags = { Name = "k3s" }
}
resource "aws_network_interface" "k3s_network_interface" {
  count           = 3
  subnet_id       = var.subnet_id
  security_groups = [aws_security_group.k3s_security_group.id]
  tags            = { Name = "k3s-network-interface" }
}
resource "aws_eip" "k3s_static_ip" {
  count             = 3
  instance          = aws_instance.k3s[count.index].id
  network_interface = aws_network_interface.k3s_network_interface[count.index].id
  vpc               = true
}

output "k3s_static_ip" {
  value = aws_eip.k3s_static_ip[*].public_ip
}

# Inbound rules for K3s server nodes
# https://docs.k3s.io/kr/installation/requirements#inbound-rules-for-k3s-server-nodes
resource "aws_security_group" "k3s_security_group" {
  name   = "k3s-security-group"
  vpc_id = var.vpc_id

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 2379
    to_port     = 2379
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 2380
    to_port     = 2380
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 6443
    to_port     = 6443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 10250
    to_port     = 10250
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 51820
    to_port     = 51820
    protocol    = "udp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 51821
    to_port     = 51821
    protocol    = "udp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # VXLAN ports: should not be exposed to public
  ingress {
    from_port   = 8472
    to_port     = 8472
    protocol    = "udp"
    cidr_blocks = [var.vpc_cidr]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Install K3s #

curl -sfL https://get.k3s.io | sh -
  • Kubectl config file: /etc/rancher/k3s/k3s.yaml
  • K3S_TOKEN: /var/lib/rancher/k3s/server/node-token
  • kubectl command: sudo k3s kubectl ...

Provide control plane’s URL as K3S_URL. This let k3s to set as a agent node cannot execute kubectl when k3s agent is being added.

curl -sfL https://get.k3s.io | \
K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -

There are far more options(parameters) to install K3s. Please refer to https://docs.k3s.io/cli/server. However, the downside of this approach is that it is not easy to configure its options dynamically (while K3s is running).