Roadmap 2023
tip
Databend 2023 Roadmap 现已 🚀 ，同步自 #9448
经过 2022 年整整一年的研发，Databend 的能力愈发完备和稳定性大幅提高。在用户的支持下，Databend 在一些业务的生产环境中落地。 Databend 帮助他们大幅 降低 成本和运维复杂度。
以下是 Databend 2023 Roadmap（供大家讨论）。
See also:
Main tasks
Features
| Task | Status | Comments |
|---|---|---|
| Update | IN PROGRESS | |
| Merge | PLAN | |
| Alter table | IN PROGRESS | |
| Window function | PLAN | |
| Lambda function and high-order functions | PLAN | |
| TimestampTz data type | PLAN | |
| Decimal data type | PLAN | |
| Materialized view | PLAN | |
| Support SET_VAR hints#8833 | PLAN | |
| Parquet reader | PLAN | |
| Distributed COPY | PLAN | |
| JSON indexing | PLAN | |
| DataFrame | PLAN | |
| Data Sharing(community version) | IN PROGRESS | |
| Concurrent query enhance | PLAN |
Improvements
| Task | Status | Comments |
|---|---|---|
| New expression#9411 | IN PROGRESS | |
| Error message | PLAN |
Planner
| Task | Status | Comments |
|---|---|---|
| Scalar expression normalization | PLAN | |
| Column constraint framework | PLAN | |
| Functional dependency framework#7438 | PLAN | |
| Join reorder | IN PROGRESS | |
| CBO for distributed plan | PLAN | |
| Support TPC-DS | PLAN | |
| Support optimization tracing | PLAN | Easy to debug/study. |
缓存
| Task | Status | Comments |
|---|---|---|
| Unified cache layer | IN PROGRESS | |
| Meta data cache | IN PROGRESS | |
| Index data cache | IN PROGRESS | |
| Block data cache | PLAN |
Data Storage
| Task | Status | Comments |
|---|---|---|
| Fuse engine re-clustering | PLAN | |
| Fuse engine orphan data cleanup | PLAN | |
| Fuse engine segment tree | PLAN | Support large dataset(PB) in one table |
LakeHouse
| Task | Status | Comments |
|---|---|---|
| Apache Hive | IN PROGRESS | |
| Apache Iceberg | IN PROGRESS | |
| Querying external storage(Parquet) | IN PROGRESS |
Distributed Query Execution
| Task | Status | Comments |
|---|---|---|
| Visualized profiling | IN PROGRESS | |
| Aggregation spilling | IN PROGRESS |
Resource Quota
| Task | Status | Comments |
|---|---|---|
| Session-level quota control (CPU/Memory) | IN PROGRESS | |
| User-level quota control (CPU/Memory) | PLAN |
Integrations
| Task | Status | Comments |
|---|---|---|
| Dbt integration | IN PROGRESS | |
| Airbyte integration | IN PROGRESS | |
| Datadog Vector integrate with Rust-driver | IN PROGRESS | |
| Datax integrate with Java-driver | IN PROGRESS | |
| CDC with Flink | PLAN | |
| CDC with Kafka | PLAN |
Meta
| Task | Status | Comments |
|---|---|---|
| Jepsen test | IN PROGRESS |
Testing
| Task | Status | Comments |
|---|---|---|
| SQLlogic Test | IN PROGRESS | Supports more test cases |
| SQLancer Test | IN PROGRESS | Supports more type and more cases |
| Fuzzer Test | PLAN |