This Week in RisingLight (Feb 16)

[skyzh]

Tutorial

For the introduction part (world1), we are halfway through the documentations.

For the storage part, we have a new design idea for storage tutorial: we will provide 1-2 parts to bootstrap a very simple storage engine, and refine it to be a real column store in the remaining parts.

Storage

In this week, we refactored the storage engine to make the code simpler and make the execution more efficient.

• storage: support blob encoding on disk (storage: support blob encoding on disk #461)
• feat(storage): support scan row handler only (feat(storage): support scan row handler only #447)
• refactor(storage): merge next_batch_inner and next_batch_inner_with_filter (refactor(storage): merge next_batch_inner and next_batch_inner_with_filter in rowset_iterator #424)
• refactor(storage): build rowset while appending data in ColumnMemTable (refactor(storage): build rowset while appending data in ColumnMemTable #414)

Computing (Optimizer, Planner, Executor)

We introduced TopN executor and made some optimizations to improve performance.

• executor: support TopN (executor: support TopN #446)
• executor: hash join support multi columns as hash key (executor: hash join support multi columns as hash key. #443)
• refactor(executor): poll concurrently with tokio::try_join! in join executors. (refactor(executor): poll concurrently with tokio::try_join! in join executors. #437)
• feat(array): introduce DataChunkBuilder and split chunks in HashJoinExecutor (feat(array): introduce DataChunkBuilder and split chunks in HashJoinExecutor #466)

Miscellaneous

We also did a lot in the interactive shell to make the output messages more human-friendly.

• human-readable message for create & insert & explain (human-readable message for create & insert & explain #440)
• cli: improve drop message (cli: improve drop and delete message #457)
• refactor(executor): use plan node to infer column name (refactor(executor): use plan node to infer column name #458)

Interesting Things...

• We need a new name for our storage engine! If you have any idea, feel free to comment on A new name for storage engine? #470
• We might hold a public RisingLight Scrum Meeting starting from next week (if there are topics to share)! See readme for more information.
• Let's welcome new RisingLight project members: @xxchan and @zzl200012!