Cloud native programming golang microservice based 25 pdf

Cloud Native programming with Golang

Develop microservice-based high performance web apps for the cloud with Go

Mina Andrawos
Martin Helmich

BIRMINGHAM - MUMBAI



Cloud Native programming with Golang
Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the publisher, except
in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express
or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held
liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.

First published: December 2017

Production reference: 1261217

Published by Packt Publishing Ltd.

Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.

ISBN 978-1-78712-598-8

www.packtpub.com


Credits
Authors
Copy Editor
Mina Andrawos
Dhanya Baburaj
Martin Helmich

Reviewer

Project Coordinator

Jelmer Snoeck

Sheejal Shah

Commissioning Editor
Aaron Lazar

Proofreader
Safis Editing

Acquisition Editor

Indexer

Nitin Dasan

Francy Puthiry

Content Development Editor

Graphics

Sreeja Nair

Jason Monteiro

Technical Editor
Prashant Mishra

Production Coordinator
Arvindkumar Gupta


About the Authors
Mina Andrawos is an experienced engineer who has developed deep experience in Go from using it
personally and professionally. He regularly authors articles and tutorials about the language, and also
shares Go's open source projects. He has written numerous Go applications with varying degrees of
complexity.
Other than Go, he has skills in Java, C#, Python, and C++. He has worked with various databases and

software architectures. He is also skilled with the agile methodology for software development.
Besides software development, he has working experience of scrum mastering, sales engineering, and
software product management.
For Nabil, Mervat, Catherine, and Fady.
Thanks to all my family for their amazing support, and continuous encouragement.

Martin Helmich studied computer science at the University of Applied Sciences in Osnabrück and
lives in Rahden, Germany. He works as a software architect, specializing in building distributed
applications using web technologies and Microservice Architectures. Besides programming in Go,
PHP, Python, and Node.js, he also builds infrastructures using configuration management tools such
as SaltStack and container technologies such as Docker and Kubernetes.
He is an Open Source enthusiast and likes to make fun of people who are not using Linux. In his free
time, you'll probably find him coding on one of his open source pet projects, listening to music, or
reading science-fiction literature.


About the Reviewer
Jelmer Snoeck is a software engineer with a focus on performance, reliability, and scaling. He's
very passionate about open source and maintains several open source projects. Jelmer comes from a
Ruby background and has been working with the Go language since 2014. He's taken a special
interest in containers and Kubernetes, and is currently working on several projects to help with the
deployment flow for these tools. Jelmer understands how to operate and scale distributed systems and
is excited to share his experience with the world.


www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files
available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you
are entitled to a discount on the eBook copy. Get in touch with us at for more

details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free
newsletters and receive exclusive discounts and offers on Packt books and eBooks.

/>
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and
video courses, as well as industry-leading tools to help you plan your personal development and
advance your career.


Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser


Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To
help us improve, please leave us an honest review on this book's Amazon page at
/dp/178712598X.
If you'd like to join our team of regular reviewers, you can e-mail us at We
award our regular reviewers with free eBooks and videos in exchange for their valuable feedback.
Help us be relentless in improving our products!


Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for

Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions

1.

Modern Microservice Architectures
Why Go?
Basic design goals
Cloud service models
Cloud application architecture patterns
The twelve-factor app
What are microservices?
Deploying microservices
REST web services and asynchronous messaging
The MyEvents platform
Summary

2.

Building Microservices Using Rest APIs
The background
So, what are microservices?
Microservices internals
RESTful Web APIs

Gorilla web toolkit
Implementing a Restful API
Persistence layer
MongoDB
MongoDB and the Go language
Implementing our RESTful APIs handler functions
Summary

3.

Securing Microservices
HTTPS
Symmetric cryptography
Symmetric-key algorithms in HTTPS
Asymmetric cryptography
Asymmetrical cryptography in HTTPS
Secure web services in Go
Obtaining a certificate
OpenSSL
generate_cert.go
Building an HTTPS server in Go


Summary

4.

Asynchronous Microservice Architectures Using Message Queues
The publish/subscribe pattern
Introducing the booking service

Event collaboration
Implementing publish/subscribe with RabbitMQ
The Advanced Message Queueing Protocol
RabbitMQ quickstart with Docker
Advanced RabbitMQ setups
Connecting RabbitMQ with Go
Publishing and subscribing to AMQP messages
Building an event emitter
Building an event subscriber
Building the booking service
Event sourcing
Implementing publish/subscribe and event sourcing with Apache Kafka
Kafka quickstart with Docker
Basic principles of Apache Kafka
Connecting to Kafka with Go
Publishing messages with Kafka
Consuming messages from Kafka
Summary

5.

Building a Frontend with React
Getting started with React
Setting up Node.js and TypeScript
Initializing the React project
Basic React principles
Kick-starting the MyEvents frontend
Implementing the event list
Bringing your own client
Building the event list components

Enabling CORS in the backend services
Testing the event list
Adding routing and navigation
Implementing the booking process
Summary

6.

Deploying Your Application in Containers
What are containers?
Introduction to Docker
Running simple containers
Building your own images
Networking containers
Working with volumes
Building containers
Building containers for the backend services
Using static compilation for smaller images
Building containers for the frontend


Deploying your application with Docker Compose
Publishing your images
Deploying your application to the cloud
Introduction to Kubernetes
Setting up a local Kubernetes with Minikube
Core concepts of Kubernetes
Services
Persistent volumes
Deploying MyEvents to Kubernetes

Creating the RabbitMQ broker
Creating the MongoDB containers
Making images available to Kubernetes
Deploying the MyEvents components
Configuring HTTP Ingress
Summary

7.

AWS I – Fundamentals, AWS SDK for Go, and EC2
AWS fundamentals
The AWS console
AWS command-line interface (CLI)
AWS regions and zones
AWS tags
AWS Elastic Beanstalk
AWS services
AWS SDK for Go
Configuring the AWS region
Configuring AWS SDK authentication
Creating IAM Users
Creating IAM Roles
The fundamentals of the AWS SDK for Go
Sessions
Service clients
Native datatypes
Shared configuration
Pagination methods
Waiters
Handling Errors

Elastic Compute Cloud (EC2)
Creating EC2 instances
Accessing EC2 instances
Accessing EC2 instances from a Linux or macOS machine
Accessing EC2 from Windows
Security groups
Summary

8.

AWS II–S3, SQS, API Gateway, and DynamoDB
Simple Storage Service (S3)
Configuring S3
Simple Queue Service (SQS)
AWS API gateway


DynamoDB
DynamoDB components
Attribute value data types
Primary keys
Secondary indexes
Creating tables
The Go language and DynamoDB
Summary

9.

Continuous Delivery
Setting up your project

Setting up version control
Vendoring your dependencies
Using Travis CI
Deploying to Kubernetes
Using GitLab
Setting up GitLab
Setting up GitLab CI
Summary

10.

Monitoring Your Application
Setting up Prometheus and Grafana
Prometheus's basics
Creating an initial Prometheus configuration file
Running Prometheus on Docker
Running Grafana on Docker
Exporting metrics
Using the Prometheus client in your Go application
Configuring Prometheus scrape targets
Exporting custom metrics
Running Prometheus on Kubernetes
Summary

11.

Migration
What is a monolithic application?
What are microservices?
Migrating from monolithic applications to microservices

Humans and technology
Cutting a monolithic application to pieces
How do we break the code?
Glue code
Microservices design patterns
Sacrificial architecture
A four-tier engagement platform
Bounded contexts in domain-driven designs
Data consistency
Event-driven architecture for data consistency
Events sourcing
CQRS
Summary


12.

Where to Go from Here?
Microservices communications
Protocol buffers
GRPC
More on AWS
DynamoDB streams
Autoscaling on AWS
Amazon Relational Database Service
Other cloud providers
Microsoft Azure
Google Cloud Platform
OpenStack
Running containers in the cloud

Serverless architectures
Summary


Preface
Cloud computing and microservices are two very important concepts in modern software
architecture. They represent key skills that ambitious software engineers need to acquire in order to
design and build software applications capable of performing and scaling. Go is a modern crossplatform programming language that is very powerful yet simple; it is an excellent choice for
microservices and cloud applications. Go is gaining increasing popularity and becoming a very
attractive skill.
The book will take you on a journey into the world of microservices and cloud computing with the
help of Go. It will start by covering the software architectural patterns of cloud applications as well
as practical concepts regarding how to scale, distribute, and deploy those applications. From there,
the book will dive deep into the techniques and design approaches that are needed for writing
production-level microservices and their deployment into typical cloud environments.
After completing this book, you will have learned how to write effective production-grade
microservices that are deployable to the cloud, practically understand the world of Amazon Web
Services, and know how to build non-trivial Go applications.


What this book covers
, Modern Microservice Architectures, opens the book by describing typical features of
cloud-based applications and microservice architectures. We will also establish requirements and a
high-level architecture for a fictional application that will serve as a continuous example over the
following chapters of this book.
Chapter 1

, Building Microservices Using REST APIs, discusses how to build modern microservices
with the Go language. We will cover important and non-trivial topics. By the end of this chapter, you
will have enough knowledge to build microservices that can expose RESTFul APIs, support

persistence, and can effectively communicate with other services.
Chapter 2

, Securing Microservices, shows you how to secure your microservices. You will get to
learn about how to handle certificates and HTTPS in the Go language.
Chapter 3

, Asynchronous Microservice Architectures, presents how to implement an asynchronous
microservice architecture using message queues. For this, we will give an overview on established
message queuing software, such as RabbitMQ and Apache Kafka, and present Go libraries to
integrate these components into your software. We will also discuss architectural patterns such as
Event Collaboration and Event Sourcing that work well together with asynchronous architectures.
Chapter 4

, Building a Frontend with React, takes a small detour from the Go world into the JavaScript
world and shows you how to build a web frontend for the microservice-based project using the React
framework. For this, we will give a short overview over the basic architectural principles of React
and how to build a React-based frontend for existing REST APIs.
Chapter 5

, Deploying Your Application in Containers, shows how to deploy Go applications in a
portable and reproducible way using application containers. You will learn to install and using
Docker and how to build custom Docker images for your own Go applications. Furthermore, we will
describe how to use the Kubernetes orchestration engine to deploy containerized applications in
large-scale cloud environments.
Chapter 6

, AWS – Fundamentals, AWS SDK for Go and AWS EC2, is the first of two chapters to cover
the AWS ecosystem. In this chapter, we will cover AWS in practical details. You will get exposed
to several important concepts like how to setup AWS server instances , how to utilize the AWS API

features, and how to write Go applications that are capable of interacting with AWS.
Chapter 7

, AWS – S3, SQS, API Gateway, and DynamoDB, continues to cover the AWS ecosystem in
more detail. You will dive deeper into popular services in the AWS world. By the end of this
chapter, you will have enough knowledge to build non-trivial Go cloud applications using the powers
of Amazon Web Services.
Chapter 8

, Continuous Delivery, describes how to implement a basic Continuous Delivery pipeline for

Chapter 9


your Go applications. For this, we will describe the basic principles of CD and how to implement a
simple pipeline using tools such as Travis CI and Gitlab. We will use Docker images as deployment
artifacts and deploy these images into a Kubernetes cluster, thus building on the topics and skills
covered in Chapter 4, Asynchronous Microservice Architectures.
, Monitoring Your Application, shows you how to monitor your microservice architecture
using Prometheus and Grafana. We will cover the basic architecture of Prometheus and describe how
to set up a Prometheus instance using Docker. Also, you will learn how to adjust your Go
applications to expose metrics that can be scraped by Prometheus. We will also describe how to set
up a graphical user interface for Prometheus using Grafana.
Chapter 10

, Migration, covers practical factors and approaches to consider when migrating from
legacy monolithic applications into modern microservices cloud-ready applications.
Chapter 11

, Where to Go from Here?, shows you where to continue the learning journey from here. It

will cover other modern cloud-related technologies that deserve to be explored, such as alternative
communication protocols, other cloud providers, and new architectural paradigms that might be the
next big thing.
Chapter 12


What you need for this book
For this book, you should have some basic knowledge of the Go programming language (if you're still
looking to get started with Go, we can recommend the book Learning Go Programming by Vladimir
Vivien, also published by Packt). To run the code examples provided in this book, you will also need
a working Go SDK on your local machine (Go 1.7 or newer). Head to for
download and installation instructions.
For many of the practical examples in the book, you will need a working Docker installation
(although previous experience in working with Docker is not required). Take a look at k
er.com/community-edition for download and installation instructions.
For working with React in Chapter 5, Building a Frontend With React, you will also need some basic
knowledge in JavaScript programming and a working Node.JS installation on your local machine.
You can download the current release of Node.JS from />

Who this book is for
This book is targeted at Go developers who want to build secure, resilient, robust, and scalable
applications that are cloud native. Some knowledge of web services and web programming should be
sufficient to get you through the book.


Conventions
In this book, you will find a number of text styles that distinguish between different kinds of
information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames,
dummy URLs, user input, and Twitter handles are shown as follows: "The react-router-dom package

adds a few new components to our application."
A block of code is set as follows:
import * as React from "react";
import {Link} from "react-router-dom";
export interface NavigationProps {
brandName: string;
}
export class Navigation extends React.Component<NavigationProps, {}> {
}

Any command-line input or output is written as follows:
$ npm install --save react-router-dom
$ npm install --save-dev @types/react-router-dom

New terms and important words are shown in bold. Words that you see on the screen, for example,
in menus or dialog boxes, appear in the text like this: "For this, click on the Create Repository after
logging in and choose a new name for your image."
Warnings or important notes appear in a box like this.

Tips and tricks appear like this.


Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you
liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really
get the most out of.
To send us general feedback, simply e-mail , and mention the book's title in the
subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to
a book, see our author guide at www.packtpub.com/authors .

Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the
most from your purchase.


Downloading the example code
You can download the example code files for this book from your account at . If
you purchased this book elsewhere, you can visit and register to have the
files e-mailed directly to you.
You can download the code files by following these steps:
1.
2.
3.
4.
5.
6.
7.

Log in or register to our website using your e-mail address and password.
Hover the mouse pointer on the SUPPORT tab at the top.
Click on Code Downloads & Errata.
Enter the name of the book in the Search box.
Select the book for which you're looking to download the code files.
Choose from the drop-down menu where you purchased this book from.
Click on Code Download.

You can also download the code files by clicking on the Code Files button on the book's webpage at
the Packt Publishing website. This page can be accessed by entering the book's name in the Search

box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest
version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at />rogramming-with-Golang. We also have other code bundles from our rich catalog of books and videos
available at Check them out!


Downloading the color images of this
book
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this
book. The color images will help you better understand the changes in the output. You can download
this file from />.pdf.