Launching an Single Server Instance of Gitlab using AWS CloudFormation
 |
 |
 |
For more information on Gitlab, visit gitlab.com
Gitlab CFT Description
This article will walk though the contents, and usage of the included CloudFormation template that launches a single server instance of Gitlab. Single server instances, include Gitlab, PostgreSQL as the database, and Redis for session storage all wrapped into a single EC2 instance. Running the CloudFormation template will yield a fully functional single server instance installed, configured, and ready for login.
The CloudFormation template that we will be using in this tutorial instantiates the following resources:
- Gitlab EC2 Instance with Gitlab, PostgreSQL, and Redis installed
- Additional 25GB EBS volume mounted to /var/opt/gitlab on the EC2 Instance
- Firewall configured with --add-service=ssh, http, https, smtp
- Security Group Created and applied to allow TCP/SSH, TCP/HTTP, and TCP/HTTPS open to the IP address or IP Range specified in the SSHLOCATION parameter.
Gitlab CFT Pre-Requisites
You will need the following in order to utilize the content of this article:
1. Active AWS Account:
You must have an active AWS account, with enough access to be able to create/update/delete CloudFormation stacks and EC2 instances.
Stack Deployment Setup
1. Log into your AWS account:
Open a browser window and visit the AWS Console Page
2. Locate and navigate to CloudFormation:
From the top left side of the navigational menu bar, click on the Services menu, and then choose CloudFormation by either navigating to the 
section of the listed services, or by typing the first few letters of the service name in the search box, and then choosing it from the filtered list.
3. Copy CF Template:
We will use the following CloudFormation template in order to launch the Gitlab instance. Copy the CloudFormation template below and save it locally on your drive as gitlab_cf_deployment.yaml. If you are using a remote host to run this lab, then copy the template to your clipboard, and then on the build host, paste the template into a file using vim /media/gitlab_cf_deployment.yaml, i, paste, and save the file by typing esc, :wq!.
AWSTemplateFormatVersion: "2010-09-09" # Description of what this CloudFormation Template is going to produce Description: AWS CloudFormation Gitlab Template to create a Stand Alone Gitlab server stack using a single EC2 instance # Define Parameter Variables that will be used throughout this CloudFormation template. Parameters: GitlabURL: Description: The URL that will be used to access Gitlab, and the URL that Gitlab will be listening for. Type: String Default: http://gitlab.example.com InstanceType: Description: Type of EC2 instance to launch for the server. Only Compute type nodes are currently specified. Type: String Default: t2.medium ConstraintDescription: Must be a valid EC2 instance type AllowedValues: - t2.nano - t2.micro - t2.small - t2.medium - t2.large - t2.xlarge - t2.2xlarge - m4.large - m4.xlarge - m4.2xlarge - m4.4xlarge - m4.10xlarge - m4.16xlarge - m5.large - m5.xlarge - m5.2xlarge - m5.4xlarge - m5.12xlarge - m5.24xlarge VpcId: Description: VPC ID that this stack will be launched in. Type: AWS::EC2::VPC::Id AllowedPattern: "[a-z0-9-]*" SubnetId: Description: VPC Subnet that this stack will be launched in. Type: AWS::EC2::Subnet::Id AllowedPattern: "[a-z0-9-]*" KeyName: Description: Name of an existing EC2 KeyPair to enable SSH access to the instance(s). Type: AWS::EC2::KeyPair::KeyName ConstraintDescription: Must be the name of an existing EC2 KeyPair SSHLocation: Description: The source IP address (/32) or source IP address range (x.x.x.x/x) that will be allowed to SSH to the EC2 instances Type: String MinLength: 9 MaxLength: 18 Default: 0.0.0.0/0 AllowedPattern: "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})" ConstraintDescription: Must be a valid IP CIDR range of the form x.x.x.x/x InstanceTagName: Description: Instance Name tag that will be used to define the Name of the instance resource(s) Type: String Default: Gitlab # Create an easy mapping, simply mapping the region selected to the appropriate Amazon Linux 2 AMI # AMI ID's Updated 8/20/18 Mappings: RegionMap: 'us-east-1': AMI: 'ami-04681a1dbd79675a5' 'us-east-2': AMI: 'ami-0cf31d971a3ca20d6' 'us-west-1': AMI: 'ami-0782017a917e973e7' 'us-west-2': AMI: 'ami-6cd6f714' # Define Resources that will be launched via this template Resources: # EC2 Server Instance Definition GitlabInstance: Description: Gitlab Standalone EC2 Instance running Gitlab, Redis, and PostgreSQL for the data storage. Type: AWS::EC2::Instance Metadata: AWS::CloudFormation::Init: # configSets is used when there are multiple configs that you want to run, for multiple instances. If not needed then just config (default) is adequate. configSets: default: [mountVolume, config, gitlabConfig] # This configSet will define how we handle the additional EBS volume that we create with the Instance. We will mount the volume in /var/opt/gitlab mountVolume: commands: 01_mkdir: command: sudo mkdir -p /var/opt/gitlab 02_fdisk: command: echo -e "o\nn\np\n1\n\n\nw" | sudo fdisk /dev/sdb 03_wait: command: sleep 3 04_mkfs: command: sudo mkfs.ext4 /dev/sdb1 05_disk_label: command: e2label /dev/sdb1 GITLAB 06_fstab: command: echo -e "LABEL=GITLAB /var/opt/gitlab ext4 defaults 0 0" >> /etc/fstab 07_mount: command: mount -a # This configSet will perform the actual installation of Gitlab gitlabConfig: packages: yum: curl: [] policycoreutils-python: [] postfix: [] commands: 01_gitlab_repo: command: curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash 02_gitlab_install: command: !Sub | sudo export EXTERNAL_URL='${GitlabURL}'; yum install -y gitlab-ce 03_gitlab_configure: command: gitlab-ctl reconfigure 04_firewall: command: sudo firewall-cmd --permanent --add-service=ssh --add-service=http --add-service=https --add-service=smtp; sudo firewall-cmd --reload services: sysvinit: postfix: enabled: true ensureRunning: true # Default Config, which handles installing the firewall, and CFN components to talk back to CloudFormation config: packages: yum: firewalld: [] commands: 01_update: command: yum -y update files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/cfn-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.GitlabInstance.Metadata.AWS::CloudFormation::Init action=/opt/aws/bin/cfn-init --verbose --stack=${AWS::StackName} --region=${AWS::Region} --resource=GitlabInstance runas=root services: sysvinit: cfn-hup: enabled: true ensureRunning: true files: - '/etc/cfn/cfn-hup.conf' - '/etc/cfn/hooks.d/cfn-auto-reloader.conf' firewalld: enabled: true ensureRunning: true # Properties of the Instance that we are launching. Here we define things like EBS volumes, SG's, The AMI used, etc.. Properties: # Create the 25GB EBS volume that we will use for /var/opt/gitlab BlockDeviceMappings: - DeviceName: /dev/sdb Ebs: DeleteOnTermination: false VolumeType: gp2 VolumeSize: 25 # Pull the Image or AMI from the RegionMap Map we defined earlier ImageId: !FindInMap - RegionMap - !Ref 'AWS::Region' - AMI # Pull the Instance Type, Subnet, KeyName, etc from the Parameters we defined earlier InstanceType: !Ref InstanceType NetworkInterfaces: - AssociatePublicIpAddress: true DeviceIndex: 0 DeleteOnTermination: true SubnetId: !Ref SubnetId # This defines the SG that we will place on the ENI, We will create the SG after the instance resource definition GroupSet: - !Ref ServerSecurityGroup KeyName: !Ref KeyName Tags: - Key: Name Value: !Ref InstanceTagName # Use the user data to instantiate the cfn service, which will report back to CloudFormaton once the instance is set up UserData: Fn::Base64: !Sub | # No more Fn::Join needed #!/bin/bash -x /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource GitlabInstance --region ${AWS::Region} /opt/aws/bin/cfn-signal -e $? --region ${AWS::Region} --stack ${AWS::StackName} --resource GitlabInstance # Creation Policy will ensure that if the instance isn't complete within the specified window, that a rollback will occur CreationPolicy: ResourceSignal: Count: '1' Timeout: PT15M # Define the Security Group that will be appended to the ENI of the Instance we are creating. ServerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Security Group that will be used for the Gitlab instance. Open ports 22,80 and 443 VpcId: !Ref VpcId SecurityGroupIngress: - IpProtocol: tcp FromPort: 22 ToPort: 22 CidrIp: !Ref SSHLocation - IpProtocol: tcp FromPort: 80 ToPort: 80 CidrIp: !Ref SSHLocation - IpProtocol: tcp FromPort: 443 ToPort: 443 CidrIp: !Ref SSHLocation # Specify any outputs for the stack. Outputs: TemplateID: Description: 'Gitlab Single Server Template' Value: 'Gitlab Single Server Template' PublicIp: Description: 'The Public IP address of the EC2 Instance' Value: !GetAtt GitlabInstance.PublicIp ServerAddress: Description: 'The public DNS address for this instance' Value: !GetAtt GitlabInstance.PublicDnsName InstanceID: Description: 'The ID of the gitlab instance that was launched' Value: !Ref GitlabInstance
3. Validate the Template (Optional):
Once the CloudFormation template has been saved to a file on your local drive or remote instance, the template can be validated using the awscli.
aws cloudformation validate-template --template-body file://gitlab_cf_deployment.yaml
Launch Options:
The CloudFormation template above can be launched in 2 ways, via the CLI or the GUI. Both steps have been included below, but ONLY ONE of the methods needs to be performed. DO NOT LAUNCH USING BOTH METHODS to avoid duplicate stacks and/or launch errors.
Stack Deployment from the Console
From the CloudFormation console screen, click the 
button to get started. If you don't have any existing stacks deployed, then you should have been directed to the CloudFormation dashboard, where you are presented with choice options where you can either create a new stack, or a new stackset. Click Create new stack from this screen. StackSets are a different topic and will not be covered in this tutorial. If you already have pre-existing stacks deployed, then you would have been brought to your main cloudformation stack list console. From this console click the Create Stack button from the top, next to the Actions button.
No Existing Stacks:
Existing Stacks:
1. Launch Stack with Template Upload:
Once on the Select Template screen, click the second option Upload a template to Amazon S3 and click the Choose File button. In the selector box window, choose the CloudFormation template that you saved to your drive from the earlier step and then click the 
button.
2. Launch Stack with Template Editor (Optional Alternative):
If you don't want to save the CloudFormation template as a file, and upload it via S3, and would instead prefer to just copy and paste your template into the console to run it, then in the CloudFormation console window, you can also choose the Design template option. By selecting the Design Template, CloudFormation will open it's template editor, where you can click on the Template tab (1), where you can paste the template yaml or JSON syntax directly into the editor (2). Once complete, click the Create Stack button (3) in the editor to allow CloudFormation to upload and automatically launch the pasted template.
3. Define Stack Parameters:
In the Details window, Type a name for the stack, and then choose the appropriate values for the stack parameters and then click the button.
- InstanceType - Instance family type, and size that you want to use for your Gitlab instance.
- KeyName - The SSH Key pair that you want assigned to the Gitlab instance.
- VpcId - The VPC that the Gitlab instance will be launched in.
- SubnetId - The subnet(s) that the Gitlab instance will be launched in.
- GitlabURL - The URL that will be configured in the /etc/gitlab/gitlab.rb file, and will tell Gitlab what URL to listen for.
- SSHLocation - The source IP address or IP address range that will be configured in the security group for allowed ingress traffic to ports 22, 80 and 443.
- InstanceTagName - The value that will be assigned to the tag "Name" key. This will effectively be what you want the instance to be named.
4. Define Stack Options:
In the Options window, set up any key/value pairs, IAM roles or notifications that you want associated with the cluster, and then click the button.
5. Review and Launch:
In the Review window, review your selected options, and once satisfied, click the I acknowledge that AWS CloudFormation might create IAM resources. check box, and click the 
button to launch the stack.
Stack Deployment CLI
As an alternative to launching off CloudFormation stacks from the AWS console, if the desire or need to launch a stack from the CLI arises, then the following section may be used instead of the console method illustrated above.
Syntax:
aws cloudformation create-stack --stack-name {{STACK_NAME}} \ --template-body file://{{FILE_NAME}}.yaml \ --capabilities CAPABILITY_IAM \ --parameters \ ParameterKey=KeyName,ParameterValue={{KEYNAME}} \ ParameterKey=VpcId,ParameterValue={{VPCID}} \ ParameterKey=InstanceType,ParameterValue={{INSTANCE_TYPE}} \ ParameterKey=GitlabURL,ParameterValue={{GITLAB_URL}} \ ParameterKey=SSHLocation,ParameterValue={{SSH_LOCATION}} \ ParameterKey=InstanceTagName,ParameterValue={{INSTANCE_TAG_NAME}} \ ParameterKey=SubnetId,ParameterValue=\"{{SUBNETID_1_ID}},{{SUBNETID_2_ID}}\"
- InstanceType - Instance family type, and size that you want to use for your Gitlab instance.
- KeyName - The SSH Key pair that you want assigned to the Gitlab instance.
- VpcId - The VPC that the Gitlab instance will be launched in.
- SubnetId - The subnet(s) that the Gitlab instance will be launched in.
- GitlabURL - The URL that will be configured in the /etc/gitlab/gitlab.rb file, and will tell Gitlab what URL to listen for.
- SSHLocation - The source IP address or IP address range that will be configured in the security group for allowed ingress traffic to ports 22, 80 and 443.
- InstanceTagName - The value that will be assigned to the tag "Name" key. This will effectively be what you want the instance to be named.
Example Request: (Replace KeyName, VpcId, SubnetId, GitLabURL, SSHLocation, InstanceType and InstanceTagName Parameters With Your Values)
aws cloudformation create-stack --stack-name Gitlab \ --template-body file://gitlab_cf_deployment.yaml \ --capabilities CAPABILITY_IAM \ --parameters \ ParameterKey=KeyName,ParameterValue=My_AWS_Key \ ParameterKey=VpcId,ParameterValue=vpc-a12345bc \ ParameterKey=InstanceType,ParameterValue=t2.medium \ ParameterKey=GitlabURL,ParameterValue=gitlab.example.com \ ParameterKey=SSHLocation,ParameterValue=1.2.3.4/32 \ ParameterKey=InstanceTagName,ParameterValue=Gitlab \ ParameterKey=SubnetId,ParameterValue=\"subnet-12a3456b,subnet-23b4567c\"
Example Response:
{ "StackId": "arn:aws:cloudformation:us-east-2:012345678910:stack/Gitlab/123a4567-8b9c-10d1-d112-13efa1bc4d1e" }
Stack Verification
1. Monitor the Launch:
Once the create button has been pushed, CloudFormation will begin to create the stack and all stack dependencies. You can watch the progress of the launch in the CloudFormation service console window.
2 . Verify the Stack:
Once the stack has been successfully launched by showing the CREATE_COMPLETE status, test the deployment, by clicking on the Output tab and copying the either the PublicIP, or ServerAddress value from the outputs section of the CloudFormation launch window.
3 . Load Gitlab:
Once the address to Gitlab has been copied out of the CloudFormation Console Outputs section, paste the address into a browser and pressing Enter. Once pressed, your browser should load Gitlab correctly.
Finalizing your Setup
Now that CloudFormation has successfully launched the Gitlab stack, we need to perform a few administrative actions on the instance in order to get Gitlab ready for usage.
1. Set your password:
When you first complete the Gitlab initialization, the first thing that you will need is to set a system root password. From the main page, Gitlab will display the set password form. Choose a strong password, confirm the password and then press the Change Your Password button.
2. Login as root:
Once you have set the initial root password, login into Gitlab with the username of root, using the password that you just set in the previous section.
3. Create New User:
Once you login, the first task you will want to perform is to set up a new non-root user. In order to do this, go to the menu bar at the top of the page, and click on the wrench icon Admin area button.
Next, from the Admin area, click on the Users navigational item from the navigation bar on the left side of the screen. Once the screen has loaded, click on the New user button in the top right hand corner.
Next, in the New user screen, fill in the users name, username, email address, and then choose their appropriate access level, along with any other optional data about the user, and click the Create user button.
Valid E-Mail NOTE:
When creating a new user, be sure to use a legitimate email address, as the set password link will be sent via email.
After the user has been created, as the root user you can logout, using the menu item in the top right hand corner of the navigation bar at the top of the screen.
4. Set New User Password:
Check your email, and you should have received an email message similar to the one below, that will contain a link to set your new users password.
NOTE:
If the email didn't get sent properly for some reason, the user's password can be set via the Admin area by the root user.
Once the link has been clicked, you will be redirected to a set password screen that is identical to what you saw in the setting the root password section above. As before, set the user password, confirm, click on the Change your password button, then login with the new user name and newly set password.
5. Set New User MFA:
Once you are logged in, then we need to set up our Multi-Factor Security, and configure our SSH key that will be used to push code to the server. These configurations will be set up in your user profile. So from the main login screen, click the user icon in the upper right hand corner of the navigation bar at the top of the screen and then choose Settings from the drop menu.
From the user menu, on the left hand side, choose the Account option from the menu. In the Account Menu screen click the Enable two factor authentication button. On the MFA screen, register your MFA device such as Authy, or Google Authenticator by scanning the QR Code, and typing in the given PIN.
6. Set New User SSH Keys:
Now that we have our MFA set, next we need to set an SSH key that will be used with Gitlab in order to interact with the Gitlab server. Follow the steps on your platform to generate an SSH key on your workstation. Once generated, copy the public key into your clipboard. From the User Settings menu, click on SSH Keys. In the SSH Keys screen, Paste the public key, name it, and click the Add key button.
We should be all set up now to make our first commit !!
Gitlab CF Conclusion
At this point we have a fully functioning Gitlab CE, single server instance fully configured and ready for code push. This setup is ideal for learning the Gitlab platform or testing different deployment scenarios, however for a full production configuration, Gitlab should be set up in an HA configuration consisting of the Gitlab front end being spread across more then one single AZ, A shared Gitlab directory configured on a network file system such as NFS, EFS, or GlusterFS, a redundant external PostgreSQL replicated database, and an HA instance of Redis.
Gitlab CF Additional Resources
Below you can find additional resources to supplement the information found in this tutorial.
Launch Gitlab Stack
Use the button below to automatically launch this cloudformation template and create your own Gitlab Single Servier instance Stack.
CloudFormation Launch
In order to use the button above you must already be logged into your AWS account. This link will launch the template the us-east-1 region. If you wish to use a different region, then copy the link and change the region=us-east-1# portion of the URL to the appropriate region.
Gitlab CloudFormation Template Download
Download the Gitlab CloudFormation Template here
Gitlab Manual Installation
In the event that you don't want to use CloudFormation in order to setup Gitlab, the following steps can be followed to install Gitlab on any Bare Metal (BM) or Virtual Machine (VM) based instance.
RHEL, 
CentOS, & 
Amazon Linux
1. Install Required Packages:
Install the required packages via Yum
yum clean all yum install -y curl policycoreutils-python postfix
2. Install the Gitlab Repository:
Now that the pre-requisite packages are installed, its time to install Gitlab.
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash
NOTE:
The following command will install the repository for Gitlab Community Edition (CE), if you wish to install Gitlab Enterprise, then replace the above URL in the command with the Enterprise Edition (EE) URL curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash
3. Set the EXTERNAL_URL Variable:
Gitlab will look for a system variable named EXTERNAL_URL when it is installed. It will use this value to fill in the external_url parameter in its /etc/gitlab/gitlab.rb configuration file. If you don't complete this step, it can be done manually later.
export EXTERNAL_URL="gitlab.awsdocs.com"
4. Install Gitlab:
yum install -y gitlab-ce
NOTE:
The following command will install Gitlab Community Edition (CE), if you wish to install Gitlab Enterprise Edition (EE), then replace the above command with the enterprise package instead
yum install -y gitlab-ee
5. Customize the Gitlab Configuration File:
Before launching the Gitlab configuration builder, edit the /etc/gitlab/gitlab.rb file to customize how Gitlab will configure itself. The configuration file will allow you to do things like turn the snippets/pages/runners/docker-registry features on and off. It will also allow you to configure Gitlab to use external PostgreSQL/Redis services instead of installing them locally on the single instance that we have configured above. Parse through the file, edit it to meet your requirements, turning whatever services on and off that you would like, and then move on to the next step to instantiate the Gitlab instance.
vim /etc/gitlab/gitlab.rb
6. Initiate the Gitlab Configuration:
This step will launch the Chef Gitlab installer using the values set in the /etc/gitlab/gitlab.rb file that we configured in an earlier step. Using the parameters in the config file, Gitlab will configure itself to install all required components locally, instantiate externals PostgreSQL/Redis services instead of locally installing those components, and turn services on or off, such as the CI/CD runners, or built in Docker Registry. The configuration file can always be changed in the future and Gitlab can be reconfigured easily to meet your evolving needs.
gitlab-ctl reconfigure
7. Firewall Configuration:
In the event that you are using firewalld to further protect your Bare Metal/AWS Instance, the following services/ports need to be open to allow ingress requests/egress responses:
sudo firewall-cmd --permanent --add-service=ssh --add-service=http --add-service=https --add-service=smtp sudo firewall-cmd --reload
8. Start Gitlab Required Services:
Now that Gitlab is installed and configured, lets start all of the Gitlab services.
systemctl enable postfix.service
systemctl start postfix.service
Controlling Gitlab with SystemCtl
The Gitlab unit file is installed and configured automatically to start on boot
9. Controlling the Gitlab Service:
Now that the Gitlab daemon is running, we now can control Gitlab using the built in Gitlab-Ctl service commandlet. This cmdlet can be used to perform tasks such as starting, stopping, restarting, and reconfiguring Gitlab along with all of Gitlab's related services using a single command. Issuing a gitlab-ctl stop for example, will stop not only Gitlab, but also the Gitlab PostgreSQL instance, as well as the Gitlab Redis instance. gitlab-ctl reconfigure is used whenever a change to the /etc/gitlab/gitlab.rb file has been made. The reconfigure command will ensure that any changes made to the configuration file are made to Gitlab as well as all Gitlab support services.
# Start all GitLab components sudo gitlab-ctl start # Stop all GitLab components sudo gitlab-ctl stop # Restart all GitLab components sudo gitlab-ctl restart # Check all GitLab components sudo gitlab-ctl status # Reconfigure Gitlab sudo gitlab-ctl reconfigure