GreptimeDB Files

Release date: September 03, 2025

👍 Highlights

• PromQL:
• Support TQL CTE in planner (#6645, #6671)
• Perf:
• Optimize sparse encoder with a 235% performance boost (#6809)
• Introduce region pruning (#6729, #6752).
• Procedure:
• Introduce metadata reconciliation procedures to repair metadata inconsistency between metasrv and datanode.
• Remote WAL:
• Replace offset-based remote WAL pruning with size-based pruning strategy and other optimizations (#6730, #6732, #6741, #6782, #6816, #6856).

🚀 Features

• feat: add metrics for reconciliation procedures by @WenyXu in #6652
• feat(metric-engine): add metadata region cache by @WenyXu in #6657
• feat: update pgwire to 0.32 by @sunng87 in #6674
• feat: Implements an iterator to merge RecordBatches by @evenyag in #6666
• feat: able to set read preference to flownode by @MichaelScofield in #6696
• feat: new HTTP API for formatting SQL by @waynexia in #6691
• feat: Implements last row dedup strategy for flat format by @evenyag in #6695
• feat: mito region staging state by @waynexia in #6664
• feat: Projection mapper for flat schema by @evenyag in #6679
• feat: add all partition column to logical table automatically by @waynexia in #6711
• feat: add integration tests for table reconciliation procedures part1 by @WenyXu in #6705
• feat(log-query): try infer and cast type for literal value by @waynexia in #6712
• feat: introduce PeriodicTopicStatsReporter by @WenyXu in #6730
• feat(metasrv): implement topic statistics collection by @WenyXu in #6732
• feat: Store partition expr in RegionMetadata by @zhongzc in #6699
• feat: improve slow queries options deserialization by @killme2008 in #6734
• feat: predicate extractor (region prune part 1) by @waynexia in #6729
• feat: persist manifest, SST and index files to staging dir by @waynexia in #6726
• feat: mysql add prepared_stmt_cache_capacity by @sunheyi6 in #6639
• feat: Implements last-non-null dedup strategy for flat format by @evenyag in #6709
• feat: flownode grpc client to frontend tls option by @discord9 in #6750
• feat: add RateMeter for tracking memtable write throughput by @WenyXu in #6744
• feat: Implements async FlatMergeReader and FlatDedupReader by @evenyag in #6761
• feat: update opentelemetry family by @sunng87 in #6762
• feat: derive macro ToRow by @WenyXu in #6768
• feat: disable month in trigger interval expr by @fengys1996 in #6774
• feat: Implements FlatCompatBatch to adapt schema in flat format by @evenyag in #6771
• feat: region prune part 2 by @waynexia in #6752
• feat: simplify more regex patterns in promql by @waynexia in #6747
• feat: add IntoRow and Schema derive macros by @WenyXu in #6778
• feat: introduce PersistStatsHandler by @WenyXu in #6777
• feat: optimize CreateFlowData with lightweight FlowQueryContext by @aaraujo in #6780
• feat: support for custom headers in otel exporter by @sunng87 in #6773
• feat: frontend internal grpc port by @discord9 in #6784
• feat: provide plan info when flow exec by @discord9 in #6783
• feat: update dashboard to v0.11.0 by @ZonaHex in #6794
• feat(mito): list SSTs from manifest and storage by @zhongzc in #6766
• feat: bump opendal to v0.54 by @killme2008 in #6792
• feat: add limit to label value api by @sunng87 in #6795
• feat: add cli option for internal grpc by @sunng87 in #6806
• feat: add FlatConvertFormat to convert record batches in old format to the flat format by @evenyag in #6786
• feat: name label regex matcher in label values api by @sunng87 in #6799
• feat: only show prometheus logical tables for name values query by @sunng87 in #6814
• feat: update dashboard to v0.11.1 by @ZonaHex in #6824
• feat: resolve unused dependencies with cargo-udeps (#6578) by @Arshdeep54 in #6619
• feat: add support for TWCS time window hints in insert operations by @WenyXu in #6823
• feat: functions and structs to scan flat format file and mem ranges by @evenyag in #6817
• feat: add replay checkpoint to reduce overhead for remote WAL by @WenyXu in #6816
• feat(metasrv): support tls for etcd client by @codephage2020 in #6818
• feat(flow): add eval interval option by @discord9 in #6623
• feat: rename region_statistics to region_statistics_history by @WenyXu in #6837
• feat: add event ts to region manifest by @discord9 in #6751
• feat: add optional schema for Postgres metadata tables by @zqr10159 in #6764
• feat: update dashboard to v0.11.2 by @ZonaHex in #6843
• feat: flow prom ql auto sink table is also promql-able by @discord9 in #6852
• feat: make etcd store max codec size configurable by @fengjiachun in #6859
• feat: Support more key types for the DictionaryVector by @evenyag in #6855
• feat: update rate limiter to use semaphore that will block without re… by @sunng87 in #6853
• feat: update dashboard to v0.11.3 by @ZonaHex in #6864
• feat: add schema and recordbatch builder for sst entry by @zhongzc in #6841
• feat: Update parquet writer and indexer to support the flat format by @evenyag in #6866
• feat: flow full aggr only trigger on new data by @discord9 in #6880

🐛 Bug Fixes

• fix: correctly set extension range source index by @MichaelScofield in #6692
• fix: TQL CTE parser take raw query string by @waynexia in #6671
• fix: metrics without physical partition columns query push down by @discord9 in #6694
• fix: unit test about trigger labels parse by @fengys1996 in #6716
• fix: http and tql should return the same value for nuknown by @yihong0618 in #6718
• fix: update pgwire to fix windows timeout issue by @sunng87 in #6710
• fix: correct offset's symbol by @waynexia in #6728
• fix: label_join should work with unknown by @yihong0618 in #6714
• fix: two label_replace different from promql by @yihong0618 in #6720
• fix: support unknown for timestamp function by @yihong0618 in #6708
• fix: truncate manifest action compat by @discord9 in #6742
• fix: refine shadowrs dependency, remove libgit2, libz and potentally libssl by @sunng87 in #6748
• fix: partition tree's dict size metrics mismatch by @waynexia in #6746
• fix: time() function should the same as behavior prometheus by @yihong0618 in #6704
• fix: time unit mismatch in lookup_frontends function by @WenyXu in #6798
• fix: plan disorder from upgrading datafusion by @waynexia in #6787
• fix: follow promql rule for hanndling label of aggr by @waynexia in #6788
• fix: correct heartbeat stream handling logic by @WenyXu in #6821
• fix: prevent stale physical table route during procedure retries by @WenyXu in #6825
• fix: use actual buf size as cache page value size by @evenyag in #6829
• fix: gRPC auth by @shuiyisong in #6827
• fix: correct config doc by @WenyXu in #6836
• fix: no need to early lookup DNS for kafka broker by @MichaelScofield in #6845
• fix: use configured kv_client in etcd multi-transaction operations by @WenyXu in #6871
• fix: fix incorrect timestamp precision in information_schema.tables by @WenyXu in #6872
• fix(flow): promql auto create table by @discord9 in #6867
• fix: ignore reserved column IDs and prevent panic on chunk_size is zero by @WenyXu in #6882
• fix: prune intermediate dirs on index finish and region pruge by @zhongzc in #6878
• fix: initialize remote WAL regions with correct flushed entry IDs by @WenyXu in #6856
• fix: move prune_region_dir to region drop by @zhongzc in #6891

🚜 Refactor

• refactor: refine error status code mappings by @WenyXu in #6678
• refactor: unify the event recorder by @zyy17 in #6689
• refactor: use DummyCatalog to construct query engine for datanode by @zhongzc in #6723
• refactor: split node manager trait by @zhongzc in #6743
• refactor: simplify WAL pruning procedure and introduce region flush trigger by @WenyXu in #6741
• refactor: enhanced trigger interval by @fengys1996 in #6740
• refactor: change plugin option type from &[PluginOptions] to Option<&PluginOptions> for understandability by @zyy17 in #6763
• refactor: refactor admin functions with async udf by @killme2008 in #6770
• refactor: use DataFusion's UDAF implementation directly by @MichaelScofield in #6776
• refactor: simplify WAL Pruning procedure part2 by @WenyXu in #6782
• refactor(meta): refactor admin service to use modern axum handlers by @WenyXu in #6833
• refactor: add stop methods for LocalFilePurger and CompactionRegion by @zyy17 in #6848
• refactor: query config options by @killme2008 in #6781

📚 Documentation

• docs(rfc): rfc for gc worker by @discord9 in #6572
• docs: improve CONTRIBUTING.md by @killme2008 in #6698
• docs: add internal grpc ports by @discord9 in #6815
• docs(rfc): async index build by @SNC123 in #6757

⚡ Performance

• perf: improve bloom filter reader's byte reading logic by @waynexia in #6658
• perf: sparse encoder by @v0y4g3r in #6809

🧪 Testing

• test: fix sqlness hash character count by @sunng87 in #6758

⚙️ Miscellaneous Tasks

• chore: pub access layer by @evenyag in #6670
• chore: add and/or for log query by @paomian in #6681
• chore: impl cast from primitives to PathType by @v0y4g3r in #6724
• chore: prefix debug_assertion only variables with underscore by @waynexia in #6727
• chore: add u64 for EqualValue and set expr is true when filter is empty by @paomian in #6731
• chore: update datafusion family by @MichaelScofield in #6675
• ci: update dev-builder image tag by @github-actions[bot] in #6759
• ci: add Signed-off-by in update-dev-builder-version script by @daviderli614 in #6765
• ci: add is-current-version-latest check to helm-charts/homebrew-greptime bump jobs by @daviderli614 in #6772
• chore: no logging when init table_flow cache if empty by @discord9 in #6785
• chore: make internal grpc optional by @discord9 in #6789
• chore: improve error message when there are more than one time index by @killme2008 in #6796
• chore: modifying the visibility of the ScalarFunctionFactory field by @paomian in #6797
• chore: add peer address context to client error logging by @WenyXu in #6793
• chore: revert [#6763] by @zyy17 in #6800
• chore: remove unused deps by @WenyXu in #6828
• chore: update bitnami config by @daviderli614 in #6847
• chore: run pull-test-deps-images.sh before docker compose to avoid rate limit by @zyy17 in #6851
• chore: add server-side error logging to improve observability in gRPC by @WenyXu in #6846
• ci: install ssh for Android dev-builder by @MichaelScofield in #6854
• ci: update dev-builder image tag by @github-actions[bot] in #6858
• chore: enlarge max file limit to 384 by @evenyag in #6868
• chore: change encode raw values signature by @v0y4g3r in #6869
• chore: use greptime dockerhub image by @daviderli614 in #6865
• chore: update dashboard by @WenyXu in #6883
• chore: fix typo by @waynexia in #6885
• chore: fix typo by @WenyXu in #6887
• chore: disable stats persistence by default by @WenyXu in #6900

New Contributors

• @github-actions[bot] made their first contribution in #6858
• @aaraujo made their first contribution in #6780
• @sunheyi6 made their first contribution in #6639

All Contributors

We would like to thank the following contributors from the GreptimeDB community:

@Arshdeep54, @MichaelScofield, @SNC123, @WaterWhisperer, @WenyXu, @ZonaHex, @aaraujo, @codephage2020, @daviderli614, @discord9, @evenyag, @fengjiachun, @fengys1996, @github-actions[bot], @killme2008, @paomian, @shuiyisong, @sunheyi6, @sunng87, @v0y4g3r, @waynexia, @yihong0618, @zhongzc, @zqr10159, @zyy17