Uploaded image for project: 'Solution Center'
  1. Solution Center
  2. SOL-243

Resource Management

    XMLWordPrintable

    Details

    • Type: Explanation
    • Status: Published
    • Affects Version/s: EXASolution 4.2.0, EXASolution 5.0, EXASOL 6.0.0, Exasol 6.1.0
    • Fix Version/s: None
    • Component/s: EXASolution
    • Labels:
      None
    • Solution:
      Hide

      General Information

      EXASolution provides a fully automatic resource management, that distributes all available resources among all active queries withing the database.
      Within EXASolution there are four different resources available:

      1. CPU
      2. RAM
      3. Network
      4. Disks

      Details

      EXASolution attemps to use as many resources as possible by internal parallelization (multi-threading). If there is more than one active query the resource management distributes the available resources among those. By default all queries are treated equally. By using priorities (see SOL-244) the resource distribution can be influenced.
      The resource management distinguishes between "short" and "long" queries:

      • Short: queries running less than 5 seconds
      • Long: queries running more than 5 seconds

      If the number of long queries exceeds the number of (logical) cores per server times 1.5 (e.g. 36 if the servers have 24 logical cores), the resource management pauses and restarts those queries automatically. In this case, time slices of 20 seconds are provided for the long queries that are not being paused. Short queries are not affected by this.
      This concept is designed to provide an optimal throughput for mixed workloads (Short and long queries).

      Time slice allocation, especially entering PAUSED mode requires pre-emptive action by the affected process. Therefore the 20 seconds are not always an exact measure, but require query execution to reach certain checkpoints. Those checkpoints are very frequent for regular execution, but recently it was detected that for example an 'active' prepared statement that is actually waiting for more client data does not enter such a checkpoint until a data packet arrives. Until this is improved on database side, we recommend to close prepared statements whenever the application realizes it will have to wait for more data.

      Show
      General Information EXASolution provides a fully automatic resource management, that distributes all available resources among all active queries withing the database. Within EXASolution there are four different resources available: CPU RAM Network Disks Details EXASolution attemps to use as many resources as possible by internal parallelization (multi-threading). If there is more than one active query the resource management distributes the available resources among those. By default all queries are treated equally. By using priorities (see SOL-244 ) the resource distribution can be influenced. The resource management distinguishes between "short" and "long" queries: Short: queries running less than 5 seconds Long: queries running more than 5 seconds If the number of long queries exceeds the number of (logical) cores per server times 1.5 (e.g. 36 if the servers have 24 logical cores), the resource management pauses and restarts those queries automatically. In this case, time slices of 20 seconds are provided for the long queries that are not being paused. Short queries are not affected by this. This concept is designed to provide an optimal throughput for mixed workloads (Short and long queries). Time slice allocation, especially entering PAUSED mode requires pre-emptive action by the affected process. Therefore the 20 seconds are not always an exact measure, but require query execution to reach certain checkpoints. Those checkpoints are very frequent for regular execution, but recently it was detected that for example an 'active' prepared statement that is actually waiting for more client data does not enter such a checkpoint until a data packet arrives. Until this is improved on database side, we recommend to close prepared statements whenever the application realizes it will have to wait for more data.
    • Category 1:
      Database Administration - Performance

      Attachments

        Activity

          People

          • Assignee:
            CaptainEXA Captain EXASOL
            Reporter:
            CaptainEXA Captain EXASOL
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated: