Managing Microservices using Terraform, Docker, and the Cloud

Size: px

Start display at page:

Download "Managing Microservices using Terraform, Docker, and the Cloud"

Clare Marilynn Kelley
6 years ago
Views:

3 Lab Preparation This lab has set-up instructions on my github Set up for the lab during the discussion! It is socially acceptable I will not wait for you to do the setup! Feel free to treat the lab as a demo 2017 Derek C. Ashmore, All Rights Reserved 3

4 Discussion Resources This slide deck The hands-on-lab code and setup instructions The Moneta microservice (written in Java) Slide deck has hyper-links! Don t bother writing down URLs 2017 Derek C. Ashmore, All Rights Reserved 4

6 What are Microservices? No concrete definition Common microservice traits Single functional purpose Most/all changes only impact one service Not dependent on execution context loosely coupled Independent process/jvm Stateless Standard Interface (typically Web Service/REST) Analogy: Stereo system, Linux utilities 2017 Derek C. Ashmore, All Rights Reserved 6

10 Docker Is a mini VM runs a linux kernal Compare to shipping container Standard connections to outside world Supported formally by Oracle, Tomcat, Jboss, and many more Package Once, Run Anywhere! 2017 Derek C. Ashmore, All Rights Reserved 10

11 Why Docker? Docker is Win-Win Easier for OPS and system administrators All software looks the same Standard interface for disk and network resources Containers can be linked Inherently automated Easier for developers Fewer environment difference issues Less to communicate to OPS / system administrators Easy to leverage work of others (docker-hub) If you haven t tried Docker yet you should! 2017 Derek C. Ashmore, All Rights Reserved 11

12 Docker Build File Docker file for Microservice Moneta Base Image FROM java:8-jre Expose Folder VOLUME /config Download Moneta Jar Release RUN curl -SL "$MONETA_URL" -o moneta-dropwizard.jar Expose Ports EXPOSE Run It ENTRYPOINT exec java -classpath $CLASSPATH -server $JAVA_OPTS -jar moneta-dropwizard.jar server /config/moneta-dropwizard.yaml 2017 Derek C. Ashmore, All Rights Reserved 12

13 Running a Docker Image The Docker Run Command Exposes Ports Attaches Disk Passes/sets Environment Variables Allocates Memory Example from the Lab Portion export JAVA_OPTS="-Xmx768m docker pull derekashmore/moneta-dropwizard:0.9.3-alpha docker run -d -p 80:8080 -m 800m -e JAVA_OPTS -v $PWD/moneta-config:/config derekashmore/moneta-dropwizard:0.9.3-alpha 2017 Derek C. Ashmore, All Rights Reserved 13

14 Typical Microservice Install at AWS Horizontal scaling is supported Multiple copies of microservice / web application running at the same time Elastic Load Balancer distributes load across copies of your service Sticky sessions available ELB can use health checks Autoscaling Groups scale number of copies up and down based on rules you give it CPU Utilization or other metrics Autoscaling Groups distribute across availability zones for availability 2017 Derek C. Ashmore, All Rights Reserved 14

16 Network Security (con t) Public vs. Private subnets Specified by Routes and Network ACLs Public subnets can be used from the internet Web Servers typically placed here Private subnets only used within your Virtual Network Access from the internet just not possible Even if a public IP is assigned Microservices Databases Messaging traffic It s common to add SSO (for web applications) and OAUTH (for microservices) above that 2017 Derek C. Ashmore, All Rights Reserved 16

17 Security Groups Security Groups provide Inbound/Outbound rules for individual instances Think of as an assignable firewall Multiple rules per VM allowed. Easy additional layer of security No changes to applications or services needed. Examples Web Servers Allow port 80 and 443 from anywhere Web Servers Allow SSH/SFTP only from within the VPC Security Groups and be associated Financial Microservice Allow port 443 only from VMs belonging to security group FINANCIAL_SERVICE_CLIENT_SG Oracle Database Allow port 1521 only from VMs belonging to security group ORACLE_CLIENT_SG 2017 Derek C. Ashmore, All Rights Reserved 17

18 Managing Cloud Assets Objectives Managing Complexity Environment Consistency Environment Lifecycle Support Ease of change Reuse Manage risk of change Mitigate Cloud Lock-in Infrastructure as Code Reusable Infrastructure Components Leverage work/expertise of others Big problem Smaller manageable problems Change Tracking Source Control 2017 Derek C. Ashmore, All Rights Reserved 18

20 Terraform Cloud Management Open Source Very active community Extensible to any cloud vendor AWS, Azure, GCP, AliCloud, Digital Ocean, OpenStack Supported for Cloud Support products Chef, Consul, Kubernetes, Datadog 62 Providers as of April, 2017 and growing 2017 Derek C. Ashmore, All Rights Reserved 20

21 Terraform HCL Declarative Language Describe what the end product contains Terraform figures out how to get there Terraform Resources Describes deployed artifacts Network Virtual Networks, Subnets, Network ACLs, Gateways, ELB/ALB Hosts Virtual Machines, Databases Security Security groups/policies/roles/groups/users Much more 2017 Derek C. Ashmore, All Rights Reserved 21

22 Terraform Basics Declarative Programming All *.tf files loaded Terraform decides execution order No GUI All command line and text editor Top Commands Terraform plan Describes planned changes Terraform apply Makes planned changes Terraform taint Forces re-creation of a resource Terraform destroy deletes all resources 2017 Derek C. Ashmore, All Rights Reserved 22

23 Terraform Resources AWS Subnet Resource Count = 3 Three subnets created Availability Zones come from a data source (lookup) CIDR blocks are input variables Sample source 2017 Derek C. Ashmore, All Rights Reserved 23

Terraform Providers Example Provider Sample AWS source

29 Terraform State Terraform stores state Local file terraform.tfstate Teams need to manage state centrally Terraform Backends Locks so that only one person at a time can update Remote storage S3, Azure containers, Google cloud storage, etc Derek C. Ashmore, All Rights Reserved 29

31 Terraform vs. Ansible/Chef Terraform designed for infrastructure Not designed for configuration management Terraform deploys images Not good at maintaining what s on those images If deployments update existing VMs You need Ansible, Chef, or Puppet If deployments are new VMs Terraform can handle deployments too 2017 Derek C. Ashmore, All Rights Reserved 31

32 Paradigm Shift Deployment as new infrastructure New version new VMs Software versions baked into images Advantages Facilitates Canary Deployments Route53 Routing Policies Go-live operation has less risk Deploy/Backout is just a load balancer switch Disadvantages More moving parts Impossible to do manually 2017 Derek C. Ashmore, All Rights Reserved 32

33 Terraform vs CloudFormation Terraform Scripting skills translate to Azure, Google Cloud, etc. Less verbose (>50%) Data Lookups Custom Plug-ins possible Active Community Support CloudFormation Quicker to follow AWS enhancements GUI support Automatic centralized state Vendor Support 2017 Derek C. Ashmore, All Rights Reserved 33

39 Beginning Steps Establish a command prompt at terraform-hands-on-lab\terraform\deployments\terraform-lab SetUp Credential Environment Variables Windows:..\..\setkeys Mac/Linux: source../../setkeys.sh Initialize the lab terraform init 2017 Derek C. Ashmore, All Rights Reserved 39

41 Sample Java/EE Microservice Moneta Greek goddess of memory Open source: Objective: Provide a RESTful Web Service interface to a relational database Feature set: Provides generic core services Returns Json-formatted data Supports startrow and maxrows query options Supports a security call-out Built-in Dropwizard, Spring Boot, and War-file deployments Sample contract spec currently read-only (writes in progress) /moneta/topics lists topics of information E.g. Topic Customer configured /moneta/topic/customers?startrow=5&maxrows=25 /moneta/topic/customer/ Docker deployment Derek C. Ashmore, All Rights Reserved 41

Questions? Derek Ashmore: Blog: www.derekashmore.com LinkedIn: www.linkedin.

Managing Microservices Using Terraform, Docker, and the Cloud

DW2 Docker Containers Wednesday, June 6th, 2018, 11:30 AM Managing Microservices Using Terraform, Docker, and the Cloud Presented by: Derek Ashmore Asperitas Consulting Brought to you by: 350 Corporate