The Foreman. Doina Cristina Duma, cristina.aiftimiei<at>cnaf.infn.it Diego Michelotto, diego.michelotto<at>cnaf.infn.it INFN-CNAF

Size: px

Start display at page:

Download "The Foreman. Doina Cristina Duma, cristina.aiftimiei<at>cnaf.infn.it Diego Michelotto, diego.michelotto<at>cnaf.infn.it INFN-CNAF"

Pauline Farmer
5 years ago
Views:

1 The Foreman Doina Cristina Duma, cristina.aiftimiei<at>cnaf.infn.it Diego Michelotto, diego.michelotto<at>cnaf.infn.it INFN-CNAF Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno 2018

2 Outline The Foreman Introduction Architecture Smart Proxies, Config. Manager, Compute Resources Provisioning Configuration Monitoring References 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

3 Is your system management tool robust enough? Image by : opensource.com 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

Effective system management tool publishing, promoting, and managing various versions of repositories and packages deploying instances or virtual machines on bare-metal, onpremises, or a cloud

4 Effective system management tool publishing, promoting, and managing various versions of repositories and packages deploying instances or virtual machines on bare-metal, onpremises, or a cloud provider's data center real-time system checkups: updates, errata, compliance, workload, and system performance metrics for registered hosts and running services create different users, roles, manage various permission policies, Kerberos & LDAP integration & authentication running arbitrary commands on hosts or groups of hosts to perform similar activities remotely bug fixes and advisory or enhancement packages audits administrator and user activities; instantly detect what changes are made in each node, when, potential risk purchase, renewal, and extension of subscriptions through a systems management portal Help identify & automate repetitive tasks 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

5 The Foreman s realm A Typical System Life-Cycle 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

6 Foreman is A complete lifecycle management tool for physical and virtual servers. 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

7 Introduction to Foreman - what is all about - 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

8 Facts Project started in 2009 Licensed under the GPLv3 Development pushed by Red Hat Very active & helpful community 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

9 Overview Tool for provisioning of VMs & bare metal Provides config management & monitoring integration Rails & JavaScript application Exposes a web interface, REST API & CLI 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

Strong & Weak points Strong Suite Weak Spots Very flexible Somewhat steep learning curve Offers tons of features Can be quite tricky to debug an issue Active development & open community API has room

10 Strong & Weak points Strong Suite Weak Spots Very flexible Somewhat steep learning curve Offers tons of features Can be quite tricky to debug an issue Active development & open community API has room for improvement Modular setup, start small then expand Can serve as a source of truth (CMDB) Offers sometimes too many possible ways to implement a task Can be used as an ENC Proper ACL implementation Enterprise Support available (Red Hat Satellite 6) 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

11 Architecture - Overview of the different components - 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

12 Bird s Eye view 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

stack, use Passenger + nginx / Apache to run it Stores most data in a DB (SQLite, MySQL or

13 Foreman Heart of the whole stack Central instance that is responsible for providing the Web based GUI, node configurations, initial host configuration files Stores all resources & information Rails stack, use Passenger + nginx / Apache to run it Stores most data in a DB (SQLite, MySQL or PostgreSQL) Local or LDAP users for authentication 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

14 Smart Proxies Small autonomous HTTP application Exposes a REST API to provide different services Allows Foreman to control components in isolated networks located on or near a machine (reduces latencies) that performs a specific function and helps foreman orchestrate the process of commissioning a new host Also called foreman-proxy 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

15 Smart Proxies Currently supported DHCP - ISC DHCP and MS DHCP Servers DNS - Bind and MS DNS Servers Puppet - Any Puppet server from 0.24.x Puppet CA - Manage certificate signing, cleaning and autosign on a Puppet CA server Realm - Manage host registration to a realm (e.g. FreeIPA) Templates - Proxy template requests from hosts in isolated networks TFTP - Any UNIX based tftp server 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

16 Smart Proxies - DHCP Takes care of reserving the required IPs Provides IP auto-assignment Supports ISC DHCP (over OMAPI), MS DHCP & libvirt More providers can be installed or developed (e.g. InfoBlox) 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

17 Smart Proxies - DNS Update and remove DNS records automatically Takes care of A, AAAA & PTR records Supports Bind, PowerDNS, Route53, MS DNS Server, Libvirt More providers can be installed or developed (e.g. AWS53) 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

18 Smart Proxies - TFTP Provide images during PXE boot Automagically downloads kernel + initrd (installer) Prepares MAC specific config depending on the build state Fallback to default 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

19 Terminology Host Installation media Provisioning templates Partition tables 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

which puppet classes should be assigned Stores which parameters apply to which puppet classes

20 Host Foreman concept that represents a server/host/system/computer In addition to holding facts about the system, it: Stores which operating system the system should be running Stores which puppet classes should be assigned Stores which parameters apply to which puppet classes Allows you to re-provision the machine 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

21 Host 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

22 Installation media the web URL from where the installation packages can be retrieved (i.e the OS mirror) Some OS Media is pre-created for you when Foreman is first installed 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

23 Provisioning templates the core of Foreman s flexibility to deploy the right OS with the right options. several types of template + a flexible matching system to deliver different templates to different Hosts PXELinux, PXEGrub, PXEGrub2 - Deployed to the TFTP server to ensure the Host boots the correct installer with the correct kernel options (also referred to as PXE templates) Provision - The main unattended installation file; e.g. Kickstart or Preseed Finish - A post-install script used to take custom actions after the main provisioning is complete user_data - Similar to a Finish script, this can be assigned to hosts built on user_data-capable images (e.g. Openstack, EC2, etc) Script - An arbitrary script, not used by default, useful for certain custom tasks ipxe - Used in {g,i}pxe environments in place of PXELinux (do not confuse with PXE templates above) pre-created templates for the mors common Os community-templates repository 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

24 Provisioning templates 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

25 Partition tables subset of normal Provisioning Templates handled separately - an admin wants to deploy the same host template (packages, services, etc) with just a different harddisk layout to account for different servers capabilities pre-created templates for common Operating Systems editable Per-Host Partition tables Dynamic Partition tables Some operating systems (Kickstart and AutoYaST) allow the creation of partition tables via scripts 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

26 Terminology Environment Compute resources Compute profiles 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

Environment Puppet environments ( isolated groups of Puppet agent nodes ) are mapped directly into Foreman Generally used to separate classes from different types of Host allowing

27 Environment Puppet environments ( isolated groups of Puppet agent nodes ) are mapped directly into Foreman Generally used to separate classes from different types of Host allowing changes to a module to tested in one environment (e.g. development) before being pushed to another (e.g production) 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

28 Environments 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

29 Compute Resources Foreman supports creating and managing hosts on a number of virtualization and cloud services - referred to as compute resources - as well as bare metal hosts 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

30 Compute Resources 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

31 Compute profiles A way of expressing a set of defaults for VMs created on a specific compute resource that can be mapped to an operator-defined label 3 predefined profiles; 1-Small, 2- Medium, and 3-Large 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

32 Provisioning - Making deployments as easy as pie - 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

33 Provisioning Provisioning includes all the tasks required to setup a new machine Saving time isn't the main goal Enforce consistency across all deployments is key 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

34 Workflow 1. Boot the installer PXE Boot (TFTP provided by Foreman) ISO image ipxe image 2. Start the installation Tell the installer where further instructions are located Red Hat Kickstart Debian Preseed Defined as kernel parameters when loading the installer 3. Get further instructions from Foreman 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

35 Installer instructions Foreman provides templating functionality ERB templates are rendered per host Contain variables, loops, snippets, etc. See provisioning templates & partition tables Templates: Foreman provides community templates Vanilla templates are locked by default Can be deleted but some are mandatory (e.g. PXELinux global default) Partition tables are used to define the filesystem layout Provisioning, Finish, 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

36 Requirements For a complete provisioning workflow we need some resources: Architecture x86_64 Installation media (mirror) OS CentOS 7 Templates Default FS Layout, Kickstart & Finish script => Follow the Hands-on! 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

37 The Foreman - Provisioning Provision new machines or containers to (almost) anything Bare metal, ovirt, libvirt, VMware, Docker, EC2, Rackspace, Digital Ocean, OpenStack, etc. Provisioning types: PXE: via PXELinux and kickstart, preseed, AutoYAST, etc Image-based: cloning, configured over SSH or user data For virtualization provider, Foreman create VM For everything Foreman orchestrates related services through Smart Proxies: DNS DHCP / TFTP Configuration Management 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

38 Configuration - Bring order into your organization - 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

39 Structure Foreman provides different resources to organize hosts: Hostgroup Domains inherited node declaration = a high level grouping of classes that can be named and treated as a unit. treated as a template and is selectable during the creation of a new host Environments Organizations & Locations 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

40 Host Groups 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

Structure Foreman provides different resources to organize hosts: Hostgroup Domains a domain and a DNS zone as the same thing hostname.somewhere.com 0> domain is somewhere.

41 Structure Foreman provides different resources to organize hosts: Hostgroup Domains a domain and a DNS zone as the same thing hostname.somewhere.com 0> domain is somewhere.com allows Foreman to associate a puppet variable with a domain/site and automatically append this variable to all external node requests made by machines at that site Environments Organizations & Locations 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

including hosts, hostgroups, domains, subnets, and compute resources, environments disabled by default

42 Structure Foreman provides different resources to organize hosts: Organizations & Locations for managing different resources inside of a single instance and granting users specific access to those resources, including hosts, hostgroups, domains, subnets, and compute resources, environments disabled by default switch context Parameter inheritance looks like this: 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

43 Config. Management Define how a system should look like in an abstract way. Foreman provides ENC functionality Simple UI to associate hosts with recipes. Multiple Puppet environment support. Allows grouping hosts. Parameterized classes with built-in hierarchical data store. Update multiple hosts at once. Supports mainly Puppet but extendable with plugins 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

44 Ansible Ansible plugin is still the new face in town Ansible provides dynamic Foreman inventory script Roles can be assigned to hosts and hostgroups Play roles through the GUI Import and delete roles through the GUI Doc: foreman_ansible plugin documentation. 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

45 The Foreman - Configuration Natively integrated with Puppet. Chef, Salt and Ansible also available via plugin. Automatic registration & setup of clients, including auto-signing certs/keys. Defining: Classes / states Parameters / pillars Inventory data: Facts / Grains Results of configuration runs 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

46 Workflow Compute resource (ovirt), also applicable to Libvirt and VMware Network (PXE) provisioning with DHCP and TFTP orchestration DNS orchestration Salt configuration management, also applicable to Puppet 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

47 Monitoring - Collect and aggregate everything - 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

48 The Foreman - Monitoring Generic Report API with graphs/trends System Inventories Reports from runs Generic reports: ABRT, OpenSCAP Context sensitive search: Not full-text Keyword completion Works across whole application 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

49 Monitoring host statuses are useful Global - OK, Warning or Error. Sub-statuses 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

Simple search through the log. Summary emails.

50 Reports Dashboard for all your Puppet hosts. Detailed log of puppet actions. Simple search through the log. Summary s. Alerts through and API. 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

51 Reports 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

52 Inventory Automatically collects your system inventory. Easy to browse and search through your inventory. Fact base permissions. Manage sets of host by fact values. Graphs! 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

53 Inventory 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

54 More data Audit log keeps track of all changes, very handy Trends give an overview of the infrastructure 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

55 Advanced Features 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

56 Plugins Katello --Sync RPM (with snapshot), Docker, and Puppet content Discovery -- Metal-as-a-Service functionality for Foreman - Auto discovery and installation of bare metal Hooks --Behavior extension of provisioning workflows Triggered on actions: on action, do X host create/update/delete, build complete, etc. X could be anything send an Can be shell, python, ruby, etc. Column_view --GUI plugins Salt/Chef --Integration with Salt/Chef Resources Many other plugins here. 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

57 Plugins 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

58 RESTful API Most of the UI actions are available in the API. Automatic Documentation. Current stable version is API V2. Powerful search API that can be used as an alternative to store-config. Used by foreman CLI and Remote-Admin. 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

59 Hammer CLI Next Gen. CLI for Foreman A new pluggable CLI tool for Foreman and Katello - Full CRUD, Uses API V2 Modeled on git with sub-commands In development, available since Foreman /06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

60 Installation Manual installation Install manually, repository, frontend, backend, database, smart-proxy, puppet and eventually dhcp, tftp and dns. Configure each component manually Puppet installation Use masterless puppet installation method in order to apply an manifest that use theforeman/foreman and theforeman/puppet modules. Configure each component through an puppet manifest. Foreman-installer Automate installation of Foreman, Smart Proxy, Puppet master, Apache and Passanger. Use Kafo - A new Ruby-gem that provides Puppet based installer. It is in use since Foreman 1.3, It has a CLI interface with progress bar, arguments for class-parameter, answer file and logging 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

61 The Foreman Foreman is a complete lifecycle management tool for physical and virtual servers. Give system administrators the power: Easily automate repetitive tasks Quickly deploy applications Proactively manage servers Supported by huge community Used by big institutes or companies CERN, Mozilla, Ericson, DHL, BBC, Citrix, Symantec Sponsored by RedHat, Rackspace 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

62 Hands-on! 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

63 Definitions Configuration management (CM) Wikipedia - field of management that focuses on establishing and maintaining consistency of a system s or product s performance and its functional and physical attributes with its requirements, design, and operational information throughout its life. For information assurance, CM can be defined as the management of security features and assurances through control of changes made to hardware, software, firmware, documentation, test, test fixtures, and test documentation throughout the life cycle of an information system PuppetLabs the process of standardizing resource configurations and enforcing their state across IT infrastructure in an automated yet agile manner. Configuration management is critical to the success of other IT processes, including provisioning, change management, release management, patch management, compliance and security. Configuration management tools should probably be considered as an essential tool when moving into the cloud From OpenStack: Maintaining an OpenStackcloud requires that you manage multiple physical servers, and this number might grow over time. Because managing nodes manually is error-prone, we strongly recommend that you use a configuration management tool. These tools automate the process of ensuring that all of your nodes are configured properly and encourage you to maintain your configuration information (such as packages and configuration options) in a version controlled repository 05/06/18 Corso Ansible/Foreman/Puppet, Bari, 5-9 Giugno

AUTOMATION ACROSS THE ENTERPRISE

AUTOMATION ACROSS THE ENTERPRISE WHAT WILL YOU LEARN? What is Ansible Tower How Ansible Tower Works Installing Ansible Tower Key Features WHAT IS ANSIBLE TOWER? Ansible Tower is a UI and RESTful API allowing