Adaptive optimization
Understand how RawTree adapts sorting keys, projections, and compression.
Adaptive optimization
RawTree can change physical data layouts as it observes data and query patterns. This reduces the need to choose every optimization before ingestion. It does not remove the need for correct SQL, suitable permissions, or enough compute capacity.
The mechanisms below act at different stages. They do not share one enable switch, and they do not all run for every query or table.
Sorting keys follow data and filters
For a raw table without an explicit key, automatic key selection can derive a sorting key for each inserted part. Background merges can use filter statistics to select a key for the merged data.
A useful key helps a query skip ranges that cannot match its predicates. Parts can use different keys while the table evolves. Re-sorting has a cost, so configured row and byte limits affect which key a merge can adopt.
An explicit table key takes precedence over automatic selection. See Sorting keys.
Projections follow repeated query patterns
When automatic projections are enabled, an advisor periodically examines eligible query patterns and can create alternate table layouts. It considers execution time, aggregation characteristics, and configured limits.
Creation and backfill are separate. The setting for automatic projection creation does not itself enable materialization for existing parts. An additional setting controls that work. The advisor can also remove automatic projections that remain unused for the configured period.
This is background adaptation, not a promise that the next execution will use a new projection. A query must match an available layout, and that layout must benefit the plan. See Projections.
Compression follows eligible stored values
Adaptive codec selection can choose a compression codec per block during merges and mutations. It applies to eligible columns that use the default codec. The current implementation supports integer-like types.
This feature is experimental. Explicit codecs remain explicit choices, and columns whose default codec includes encryption do not use adaptive selection. Compression changes storage representation, not the logical field values or query result types.
Compression savings and query speed are different measurements. Additional codec-selection work can consume CPU even when it reduces stored bytes.
Settings and defaults
| Mechanism | Main setting | Engine default |
|---|---|---|
| Automatic sorting keys | autogenerate_primary_key | Enabled for eligible raw tables without an explicit key. |
| Automatic projection creation | enable_auto_projections | Disabled. |
| Automatic projection backfill | auto_projection_materialize_after_create | Disabled. |
| Adaptive codec selection | enable_adaptive_codec_selection | Disabled and experimental. |
These are engine defaults, not a report of a particular cluster's configuration. Cluster and table configuration can override them. The generated setting reference describes available engine controls, not which changes the public Query API permits.
Evaluate benefits and costs
Adaptation uses resources for analysis, merges, materialization, and storage. A changing workload can produce different choices from a stable repeated one. No setting guarantees that every query becomes faster.
A useful evaluation compares the same query and data range before and after a change. Plans, rows read, bytes read, and elapsed time help explain whether the physical change helped. New data and query caches can affect comparisons.
Optimization preserves SQL semantics. It cannot decide whether a numeric string and a number represent the same business value. That decision belongs in the query's treatment of dynamic fields.