We measured the ntx-builder with a large account (1M storage map entries) and realized that the current approach with the full Account in-memory is not scalable.
We should try an alternative, treating the native accounts the same way the foreign-account path already does which is using the store as an oracle, and the ntx-builder holds only the header state in memory. It requires some changes to the database, but overall reduces the amount of code. In memory we would have a PartialAccount which isn't a problem long term. This approach also prevents the divergence that we had in the past of the in memory account not matching the node one, since the store is only updated by proven blocks and all the state is read from there.
We measured the
ntx-builderwith a large account (1M storage map entries) and realized that the current approach with the fullAccountin-memory is not scalable.We should try an alternative, treating the native accounts the same way the foreign-account path already does which is using the store as an oracle, and the ntx-builder holds only the header state in memory. It requires some changes to the database, but overall reduces the amount of code. In memory we would have a PartialAccount which isn't a problem long term. This approach also prevents the divergence that we had in the past of the in memory account not matching the node one, since the store is only updated by proven blocks and all the state is read from there.