[SOL-577] Column statistics Created: 29.09.2017  Updated: 29.07.2020  Resolved: 29.07.2020

Status: Obsolete
Project: Solution Center
Component/s: EXASolution
Affects Version/s: EXASOL 6.0.0, EXASolution 5.0.0, Exasol 6.2.x
Fix Version/s: None

Type: Explanation
Reporter: Captain EXASOL Assignee: Captain EXASOL
Labels: None

Issue Links:
Causing
is caused by SOL-450 Upgrade solution 5.0.x to version 6.0 Obsolete
Related
Explanation:

Note: This solution is no longer maintained. For the latest information, please refer to our documentation:

https://community.exasol.com/t5/database-features/column-statistics/ta-p/1500

Background

Exasol database automatically computes and stores column statistics on demand, e.g. when joining tables for the first time. Those statistics consist of distinct estimates, minimum & maximum values, balancing, and other information. They are used by our query optimizer for estimation of join costs, filter sensitivities, etc.

The computation of column statistics performs a complete column scan. Therefore it might be expensive, especially for large tables and/or tables not having all data in memory. This means that a query doing statistics computation may experience a significant slow down compared to the next execution.

Column statistics are maintained as part of DML statements. They are recomputed if a significant amount of data has been changed since the last computation.

Explanation

Problem Description

Due to format changes, all column statistics are invalidated during an update from EXASOL 5.0 to EXASOL 6.0.

Required User Action

 After the update to EXASOL 6.0, we recommend to recompute the statistics for the whole database to avoid any potential unexpected performance losses. Please note that the following command is introduced with version 6.0.4 (see ---EXASOL-2110---).

ANALYZE DATABASE REFRESH STATISTICS;

Alike other multi-table statements, ANALYZE DATABASE does an implicit COMMIT after each table minimizing transaction conflicts.

Time Estimation

It is often useful to obtain a estimate on the duration of the ANALYZE DATABASE REFRESH STATISTICS statement.

The query below delivers such an estimate (measured in seconds) when run before the update (that is, on EXASOL 5.0, while statistics are still valid):

select
    cast(
        zeroifnull(
            sum(raw_object_size) / 1024 / 1024 / 150 / nproc()
        ) as dec(18, 1)
    ) as COLUMN_STATISTICS_REFRESH_SECONDS
from
    "$EXA_COLUMN_SIZES"
where
    (column_schema, column_table, column_name) in (
        select
            column_schema,
            column_table,
            column_name
        from
            "$EXA_COLUMN_STATISTICS"
        where
            -- filter does not work on 6.0 before the REFRESH
            min_value_estimate is not null  
    );

Additional References

SOL-450 Upgrade solution 5.0.x to version 6.0

SOL-520 Statistics export for support v6.0x

SOL-222 Exastatics schema

SOL-437 Downloading database usage stats

 

 

 

 

Category 1: Database Administration - Performance
Generated at Tue Dec 07 18:30:54 CET 2021 using Jira 7.13.18#713018-sha1:e1230154f8ff8cc9272975bf568fc732e806fd68.