Uploaded image for project: 'Solution Center'
  1. Solution Center
  2. SOL-633

AWS Single Node Installation via CloudFormation Template

    XMLWordPrintable

    Details

    • Type: Explanation
    • Status: Published
    • Affects Version/s: Exasol 6.1.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Solution:
      Hide

      You can easily launch an EXASOL single node database on AWS using AWS CloudFormation
      This Getting Started guide helps you get up and running with EXASOL quickly.
      It covers the following steps:

      1. Prerequisites
      2. Selecting an appropriate instance type and storage
      3. Configuring the instance and launching the deployment
      4. Logging into EXAoperation
      5. Connecting using EXAplus

      1. Prerequisites:

      In order to deploy Exasol using CloudFormation you have to subscribe to the corresponding offering in the AWS Marketplace first.

      • Sign in into the aws marketplace and subscribe to the corresponding EXASOL offering under:
      • When your subscription process has been completed you get a notification by mail (alternatively you can verify if the AMIs are available by typing "exasol" in the search field for public EC2 AMIs in the AWS Console)

      2. Selecting an appropriate instance type and storage size

      A good rule of thumb is that the RAM size of the selected instance should correspond to approximately 10% - 15 % of your raw data.

      Quick Tip
      The template is preconfigured for approximately 500GB - 600 GB of raw data (m4.4xlarge with 16 vcores, 64 GB main and 420 GB storage). If this configuration is fine for you, you might skip the rest of this section

      EXASOL supports all EC2 instances with >15GB Main Memory.
      We recommend that you use instance types of category r4,r5,m4,m4 or c5 with >30 GB main memory.

      If we assume that your raw data (uncompressed) is about 500 GB in size, the following instance types could be a good choice:

      • r4.2xlarge, 8 vcores, 61 GB main memory
      • m4.4xlarge, 16 vcores, 64 GB main memory

      The main difference between those instance type is the number of vcores: the compute power.

      Due to it's higher compute power the m4.4xlarge type will be able to process more concurrent queries compared to the r4.2xlarge instance.

      In this guide we decided to use the m4.4xlarge instance model therefore.

      Please note:

      • The sizing "rule of thumb" used above may not be appropriate for your use case. Based on your specific use case, you may need significantly more memory.
      • For more information on sizing please refer to: EXASOL Sizing and the corresponding section in the FAQ document on installation and configuration

      The data storage volumes should be properly sized in order to store the maximum recommended amount of data for this instance type, data replicas in case of a cluster configuration as well as index structures and space for temporary data.

      This template already configures the correct storage configuration for every instance type.

      When selecting the instance type and storage "r3.8xlarge-2.928TB--EBS" the optimal number of EBS drives with a total size of 2.928TB will be automatically attached to every node.

      If you would like to configure the storage independently from the instance types please use the templates in SOL-636

      3. Configuring the instance and launching the deployment using CloudFormation

      Run Cloudformation directly via:

      (use existing VPC)      (generate new VPC) 

      or

      This list contains all parameters for both kind of templates:

      Parameter Usage/Description
      Stack name Name of the AWS CloudFormation Stack
      Database Name Name of the generated database
      SYS user password Password for the exasol database administration user (SYS)
      ADMIN user Password Password for the EXAoperation system administration user (ADMIN)
      VPC CIDR Block If a new VPC is created, a valid CIDR Block has to be specified
      Database Subnet-Id (only when deploying into existing VPC) Choose a SubnetID from the dropdown menu
      Subnet CIDR Block Choose a valid CIDR Block if deplyoment into new VPC (Optional)
      Database Placement Group An existing placement group can be choosen here, else a new one is created
      Public IPs If true, associate public ip addresses to all instances
      DNS Server DNS Server (default AWS DNS Server 169.254.169.253)
      System Timezone Required timezone for the DB (default Europe/Berlin)
      License Server IP IP address of the License Server (Data Nodes IP are counting upwards from this one) (Optional)
      License Server Instance Type EC2 instance type of the license server
      AMI ID Select corresponding AMI from the AWS Marketplace
      First Data Node IP Counting upwards for the IPs of the data nodes (FirstDataNodeIP > LicenceServerIP) (Optional)
      Number of Data Nodes Number of database nodes that store data and process queries (min:2, max:64)
      Data Node Instance Type Instance type of data nodes
      Replication Factor Replication Factor defines how many copies of a data block are kept in the cluster (1 means: no redundancy)
      StandByNodes If the replication factor is > 1 then a standby node can automatically replace a failed node
      Encrypt EBS Volumes Enable encryption of block storage
      Remote Access From IP 0.0.0.0/0 if allow access from anywhere
      AWS Key Pair Choose keypair for SSH access of created instances
      License In case of a BYOL image, an already aquired license can be pasted here
      • Click next to proceed to the options view (normally no further configuration is needed here)
      • After reviewing your configuration parameters confirm the message I acknowledge that AWS CloudFormation might create IAM resources. (Remark: This message is displayed because the template creates an AWS-Lambda resource that needs IAM permissions to launch the EC2 instances)

      Now it will take some time (10-25 minutes) until the database node is available.

      Determinig the public IP of the database server

      • You may have to click the refresh button first to see your deployment in the overview list
      • In the cloud formation console wait until the Status of the deployed stack shows "CREATE_COMPLETE".
      • Select the stack and select "Resources"
      • Select the ID field of "DBServer" to jump to the virtual machine of the database server
      • The public IP address can now be found in the description area after the label "IPv4 Public IP"

      4. Logging into the administration frontend EXAoperation

      Please note: It may take a while until the web based administration application EXAopertion becomes available after deployment (this may take up to 45 minutes)

      When the installation has been finialized, you can access the administration application EXAOperation via a browser at https://<public_dns>/.
      Please confirm the digital certificate first as its validity cannot be automatically verified by the web browser ("Your connection is not secure").

      You can log into EXAoperation using the username 'admin' and the password defined for this user.
      If you can open the login page but you cannot login using the password you defined during the deployment, the installation might be still in progress. Access to EXAoperation will not be unlocked before the installation has been finished

      5. Connecting using EXAplus

      You need to first download and install EXAplus, which is the standard SQL client from Exasol. You can download EXAplus from Exasol Download Section.

      If you defined Public IPs: true during deployment, port 8563, Exasol's standard communication port can be accessed from outside AWS. This can be changed in the security group configuration.

      Open EXAplus and log in with your Sys username and password, which you created in the previous step.

       

       

       

       

      Show
      You can easily launch an EXASOL single node database on AWS using AWS CloudFormation This Getting Started guide helps you get up and running with EXASOL quickly. It covers the following steps: Prerequisites Selecting an appropriate instance type and storage Configuring the instance and launching the deployment Logging into EXAoperation Connecting using EXAplus 1. Prerequisites: In order to deploy Exasol using CloudFormation you have to subscribe to the corresponding offering in the AWS Marketplace first. Sign in into the aws marketplace and subscribe to the corresponding EXASOL offering under: the Pay-As-You-Go (PAYG) offering: https://aws.amazon.com/marketplace/pp/B01FXIPXV4 the Bring-Your-Own-License (BYOL): offering: https://aws.amazon.com/marketplace/pp/B01IVOPPN8 When your subscription process has been completed you get a notification by mail (alternatively you can verify if the AMIs are available by typing "exasol" in the search field for public EC2 AMIs in the AWS Console) 2. Selecting an appropriate instance type and storage size A good rule of thumb is that the RAM size of the selected instance should correspond to approximately 10% - 15 % of your raw data. Quick Tip The template is preconfigured for approximately 500GB - 600 GB of raw data (m4.4xlarge with 16 vcores, 64 GB main and 420 GB storage). If this configuration is fine for you, you might skip the rest of this section EXASOL supports all EC2 instances with >15GB Main Memory. We recommend that you use instance types of category r4,r5,m4,m4 or c5 with >30 GB main memory. If we assume that your raw data (uncompressed) is about 500 GB in size, the following instance types could be a good choice: r4.2xlarge, 8 vcores, 61 GB main memory m4.4xlarge, 16 vcores, 64 GB main memory The main difference between those instance type is the number of vcores: the compute power. Due to it's higher compute power the m4.4xlarge type will be able to process more concurrent queries compared to the r4.2xlarge instance. In this guide we decided to use the m4.4xlarge instance model therefore. Please note: The sizing "rule of thumb" used above may not be appropriate for your use case. Based on your specific use case, you may need significantly more memory. For more information on sizing please refer to: EXASOL Sizing and the corresponding section in the FAQ document on installation and configuration The data storage volumes should be properly sized in order to store the maximum recommended amount of data for this instance type, data replicas in case of a cluster configuration as well as index structures and space for temporary data. This template already configures the correct storage configuration for every instance type. When selecting the instance type and storage "r3.8xlarge-2.928TB--EBS" the optimal number of EBS drives with a total size of 2.928TB will be automatically attached to every node. If you would like to configure the storage independently from the instance types please use the templates in SOL-636 3. Configuring the instance and launching the deployment using CloudFormation Run Cloudformation directly via: (use existing VPC)      (generate new VPC)  or Open the AWS-Console and select the CloudFormation section. Click Create Stack Upload the corresponding template which fulfills your needs (see last section in this description) or use directly the url to the following S3-Buckets. https://s3.eu-central-1.amazonaws.com/exasol-cloudformation-templates-marketplace/exasol_single_node_cf_marketplace.json https://s3.eu-central-1.amazonaws.com/exasol-cloudformation-templates-marketplace/exasol_single_node_cf_marketplace_generate_new_VPC.json Complete the form with your desired configuration (there are already some preselected entries which can be overwritten) This list contains all parameters for both kind of templates: Parameter Usage/Description Stack name Name of the AWS CloudFormation Stack Database Name Name of the generated database SYS user password Password for the exasol database administration user (SYS) ADMIN user Password Password for the EXAoperation system administration user (ADMIN) VPC CIDR Block If a new VPC is created, a valid CIDR Block has to be specified Database Subnet-Id (only when deploying into existing VPC) Choose a SubnetID from the dropdown menu Subnet CIDR Block Choose a valid CIDR Block if deplyoment into new VPC (Optional) Database Placement Group An existing placement group can be choosen here, else a new one is created Public IPs If true, associate public ip addresses to all instances DNS Server DNS Server (default AWS DNS Server 169.254.169.253) System Timezone Required timezone for the DB (default Europe/Berlin) License Server IP IP address of the License Server (Data Nodes IP are counting upwards from this one) (Optional) License Server Instance Type EC2 instance type of the license server AMI ID Select corresponding AMI from the AWS Marketplace First Data Node IP Counting upwards for the IPs of the data nodes (FirstDataNodeIP > LicenceServerIP) (Optional) Number of Data Nodes Number of database nodes that store data and process queries (min:2, max:64) Data Node Instance Type Instance type of data nodes Replication Factor Replication Factor defines how many copies of a data block are kept in the cluster (1 means: no redundancy) StandByNodes If the replication factor is > 1 then a standby node can automatically replace a failed node Encrypt EBS Volumes Enable encryption of block storage Remote Access From IP 0.0.0.0/0 if allow access from anywhere AWS Key Pair Choose keypair for SSH access of created instances License In case of a BYOL image, an already aquired license can be pasted here Click next to proceed to the options view (normally no further configuration is needed here) After reviewing your configuration parameters confirm the message I acknowledge that AWS CloudFormation might create IAM resources. (Remark: This message is displayed because the template creates an AWS-Lambda resource that needs IAM permissions to launch the EC2 instances) Now it will take some time (10-25 minutes) until the database node is available. Determinig the public IP of the database server You may have to click the refresh button first to see your deployment in the overview list In the cloud formation console wait until the Status of the deployed stack shows "CREATE_COMPLETE". Select the stack and select "Resources" Select the ID field of "DBServer" to jump to the virtual machine of the database server The public IP address can now be found in the description area after the label "IPv4 Public IP" 4. Logging into the administration frontend EXAoperation Please note: It may take a while until the web based administration application EXAopertion becomes available after deployment (this may take up to 45 minutes) When the installation has been finialized, you can access the administration application EXAOperation via a browser at https://<public_dns>/. Please confirm the digital certificate first as its validity cannot be automatically verified by the web browser ("Your connection is not secure"). You can log into EXAoperation using the username 'admin' and the password defined for this user. If you can open the login page but you cannot login using the password you defined during the deployment, the installation might be still in progress. Access to EXAoperation will not be unlocked before the installation has been finished 5. Connecting using EXAplus You need to first download and install EXAplus, which is the standard SQL client from Exasol. You can download EXAplus from Exasol Download Section . If you defined Public IPs: true during deployment, port 8563, Exasol's standard communication port can be accessed from outside AWS. This can be changed in the security group configuration. Open EXAplus and log in with your Sys username and password, which you created in the previous step.        
    • Category 1:
      Plattform Support - EXASOL on AWS
    • Category 2:
      Cluster Administration - Installation

      Attachments

        Issue Links

          Activity

            People

            • Assignee:
              CaptainEXA Captain EXASOL
              Reporter:
              CaptainEXA Captain EXASOL
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: