Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
fa851f8
ammend based on feedback
Apr 30, 2025
27c7536
chore: empty blocks & lazy aggrgation (#2243)
tac0turtle May 2, 2025
351e1d0
remove unused
May 2, 2025
d2a6061
reset block timer when no txs are present
May 2, 2025
b9db5d6
Simplify
Manav-Aggarwal May 2, 2025
58d5dc4
Implement execValidate and insert back dataHash
Manav-Aggarwal May 2, 2025
e1dbbea
Add batchValidation in retriever
Manav-Aggarwal May 2, 2025
854af43
modify validation logic
Manav-Aggarwal May 2, 2025
ae1027b
Add DA Includer Loop
Manav-Aggarwal May 2, 2025
b441ce9
Fix test syntax
Manav-Aggarwal May 2, 2025
663f37e
Add Validate method for signed header
Manav-Aggarwal May 2, 2025
287988c
setDAIncluded for sequencers too
Manav-Aggarwal May 2, 2025
b675f3d
Fix first retriever test
Manav-Aggarwal May 3, 2025
f1e5936
Refactor retriever and fix more tests
Manav-Aggarwal May 3, 2025
ba4c23b
fixed all retriever tests
Manav-Aggarwal May 3, 2025
474c757
Add more tests to retriever and handle empty blobs
Manav-Aggarwal May 3, 2025
5b464e4
make fromProto more robust
Manav-Aggarwal May 3, 2025
debb51c
Add doc headers to add test methods in block package
Manav-Aggarwal May 3, 2025
a2b3c71
Make TestProcessNextDAHeader_MultipleHeadersAndBatches more robust
Manav-Aggarwal May 3, 2025
adb01f6
Fix from rebase
Manav-Aggarwal May 3, 2025
64f848c
fix publishBlock tests
Manav-Aggarwal May 3, 2025
ab1304e
Add daIncluderTests
Manav-Aggarwal May 3, 2025
ca16508
enable t.Parallel in all block tests
Manav-Aggarwal May 3, 2025
df22050
go mod tidy everywhere
Manav-Aggarwal May 3, 2025
2e9b229
catch errcheck in setfinal and add test for it in daIncluder
Manav-Aggarwal May 3, 2025
e555a59
fix lint
Manav-Aggarwal May 3, 2025
7c0414d
remove gci from golintci
Manav-Aggarwal May 3, 2025
3675dc6
Fix first test in sync_test
Manav-Aggarwal May 3, 2025
eef9b69
Fix all sync tests
Manav-Aggarwal May 4, 2025
770170a
fix execution_test.go
Manav-Aggarwal May 4, 2025
0ede403
Modify dummy
Manav-Aggarwal May 4, 2025
ce8e3f0
enhance daIncluder
Manav-Aggarwal May 4, 2025
8062955
fix first integration test
Manav-Aggarwal May 4, 2025
c380d28
Fix node_intergration_test
Manav-Aggarwal May 4, 2025
c3a32e8
Fix node package test
Manav-Aggarwal May 4, 2025
de5e6bd
fix lint
Manav-Aggarwal May 4, 2025
87dec50
Move batchSubmissionLogic out of sequencer
Manav-Aggarwal May 5, 2025
f03d0a5
Remove timestamp and appHash validation
Manav-Aggarwal May 5, 2025
b8194fe
Fix rebase issues
Manav-Aggarwal May 5, 2025
d1895e4
Add back isDAIncluded
Manav-Aggarwal May 5, 2025
f4475dc
Remove dataCh from retriever
Manav-Aggarwal May 5, 2025
2dcd061
add doc header for retriever
Manav-Aggarwal May 5, 2025
08d7846
random refactors
Manav-Aggarwal May 5, 2025
028e456
revert lazy aggregation adr changes and fix helper signature
Manav-Aggarwal May 5, 2025
6081ec8
ad DAHeight to NextState method
Manav-Aggarwal May 5, 2025
f7805cc
fix lint and parse flags test
Manav-Aggarwal May 5, 2025
cf34b85
Refactor
Manav-Aggarwal May 5, 2025
fe00e91
Clarify how DataHash is set
Manav-Aggarwal May 5, 2025
81944c8
fix data race issue
Manav-Aggarwal May 5, 2025
2011668
Protect GetLastState using mutex
Manav-Aggarwal May 5, 2025
5550982
Fix bug
Manav-Aggarwal May 7, 2025
45ba444
Fix lint
Manav-Aggarwal May 7, 2025
21c7231
remove channel from sequencer constructor
Manav-Aggarwal May 7, 2025
3b00beb
make datahashforempty private
Manav-Aggarwal May 7, 2025
f0d7bbc
Add DAIncluder in block manager
Manav-Aggarwal May 7, 2025
0ae49c1
fix run and update go mod
Manav-Aggarwal May 7, 2025
e83f9f5
Merge branch 'main' into manav/fix_batch_validation
julienrbrt May 14, 2025
1d299bd
fix: Refactor DA Inclusion logic (#2277)
Manav-Aggarwal May 15, 2025
61c04b6
Merge branch 'main' into manav/fix_batch_validation
julienrbrt May 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add DAIncluder in block manager
  • Loading branch information
Manav-Aggarwal committed May 7, 2025
commit f0d7bbc96c2f25e119ba2979c7afd2428a362e70
28 changes: 15 additions & 13 deletions specs/src/specs/block-manager.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,16 +121,23 @@ Among non-sequencer full nodes, all the block gossiping is handled by the block

### Block Retrieval from P2P network

For non-sequencer full nodes, Blocks gossiped through the P2P network are retrieved from the `Block Store` in `BlockStoreRetrieveLoop` in Block Manager.
Starting off with a block store height of zero, for every `blockTime` unit of time, a signal is sent to the `blockStoreCh` channel in the block manager and when this signal is received, the `BlockStoreRetrieveLoop` retrieves blocks from the block store.
It keeps track of the last retrieved block's height and every time the current block store's height is greater than the last retrieved block's height, it retrieves all blocks from the block store that are between these two heights.
For non-sequencer full nodes, blocks gossiped through the P2P network are retrieved from the `Block Store` in `BlockStoreRetrieveLoop` in Block Manager.
Starting off with a block store height of zero, for every `blockTime` unit of time, a signal is sent to the `blockStoreCh` channel in the block manager and when this signal is received, the `BlockStoreRetrieveLoop` retrieves blocks from the block store.
It keeps track of the last retrieved block's height and every time the current block store's height is greater than the last retrieved block's height, it retrieves all blocks from the block store that are between these two heights.
For each retrieved block, it sends a new block event to the `blockInCh` channel which is the same channel in which blocks retrieved from the DA layer are sent.

This block is marked as soft confirmed by the validating full node until the same block data and the corresponding header is seen on the DA layer then marked DA-included.
This block is marked as soft confirmed by the validating full node until the same block data and the corresponding header is seen on the DA layer, then it is marked DA-included.

#### About Soft Confirmations and DA Inclusions

The block manager retrieves blocks from both the P2P network and the underlying DA network because the blocks are available in the P2P network faster and DA retrieval is slower (e.g., 1 second vs 15 seconds). The blocks retrieved from the P2P network are only marked as soft confirmed until the DA retrieval succeeds on those blocks and they are marked DA included. DA included blocks can be considered to have a higher level of finality.
The block manager retrieves blocks from both the P2P network and the underlying DA network because the blocks are available in the P2P network faster and DA retrieval is slower (e.g., 1 second vs 6 seconds).
The blocks retrieved from the P2P network are only marked as soft confirmed until the DA retrieval succeeds on those blocks and they are marked DA-included.
DA-included blocks are considered to have a higher level of finality.

**DAIncluderLoop**:
A new loop, `DAIncluderLoop`, is responsible for advancing the `DAIncludedHeight` by checking if blocks after the current height have both their header and data marked as DA-included in the caches.
If either the header or data is missing, the loop stops advancing.
This ensures that only blocks with both header and data present are considered DA-included.

### State Update after Block Retrieval

Expand All @@ -144,15 +151,10 @@ The block manager stores and applies the block to update its state every time a

The communication between the block manager and executor:

* `InitChain`: using the genesis, a set of parameters, and validator set to invoke `InitChainSync` on the proxyApp to obtain initial `appHash` and initialize the state.
* `Commit`: commit the execution and changes, update mempool, and publish events.
* `InitChain`: initializes the chain state with the given genesis time, initial height, and chain ID using `InitChainSync` on the executor to obtain initial `appHash` and initialize the state.
* `CreateBlock`: prepare a block by polling transactions from mempool.
* `ApplyBlock`: validate the block, execute the block (apply transactions), validator updates, create and return updated state

The communication between the full node and block manager:

* Notify when the block is published
* Notify when the block is done lazy building
* `ApplyBlock`: validate the block, execute the block (apply transactions), validator updates, create and return updated state.
* `SetFinal`: sets the block as final when it's corresponding header and data are seen on the dA layer.

## Assumptions and Considerations

Expand Down