Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Rollbackable fields private #209

Merged
merged 69 commits into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from 68 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
5de8afb
Derive partial-eq for VmState
MarcosNicolau Aug 13, 2024
59470a7
Add initial gas param for VmState new
MarcosNicolau Aug 13, 2024
26c2b18
Update era-compiler-tester branch
MarcosNicolau Aug 13, 2024
e5cb98b
fix test bytecode_publishing
juan518munoz Aug 14, 2024
f0dce41
Store changes for get_used_contracts tests
MarcosNicolau Aug 14, 2024
c862649
adapt to zksync-era tests
juan518munoz Aug 16, 2024
bbac76d
Remove hash_map from Storage trait
MarcosNicolau Aug 16, 2024
5ca9edf
Fix increment_tx_number not cleaning transient_storage
MarcosNicolau Aug 20, 2024
01c6eb6
Fix decommit opcode
MarcosNicolau Aug 20, 2024
be5024f
Various far_call fixes
MarcosNicolau Aug 20, 2024
cdfcd92
Add stipend param to frame and decrease it in panics and reverts
MarcosNicolau Aug 20, 2024
5be7e35
Add no refund storage_read for far_call decommit_code_hash
MarcosNicolau Aug 20, 2024
29c184a
Remove rollbacks from main context
MarcosNicolau Aug 20, 2024
679d38d
Reimplement vec snapshots
MarcosNicolau Aug 20, 2024
1c9a4b2
Add external snapshot(whole vm)
MarcosNicolau Aug 20, 2024
f6f2829
Add storage cache and util functio to get storage changes from initial
MarcosNicolau Aug 20, 2024
8747004
Refactor state pub fields
MarcosNicolau Aug 20, 2024
2a28162
Add new function to get storage changes from snapshot or initial
MarcosNicolau Aug 20, 2024
6509f4a
Merge branch 'main' into zksync-era-integration-tests
MarcosNicolau Aug 20, 2024
41db1b5
Update zksync-era submodule
MarcosNicolau Aug 20, 2024
fe7a666
Update era-compiler-tester submodule
MarcosNicolau Aug 20, 2024
18c5199
Address clippy warnings
MarcosNicolau Aug 20, 2024
897eb16
More clippy warnings
MarcosNicolau Aug 20, 2024
4efc066
Update zksync-era submodule
MarcosNicolau Aug 20, 2024
b512b31
ci: add submodule step for era
fkrause98 Aug 20, 2024
b4c8773
initial commit
juan518munoz Aug 20, 2024
a5700ea
Address review comments
MarcosNicolau Aug 20, 2024
c30ea73
Add cache to storage_read
MarcosNicolau Aug 21, 2024
cd451dd
Update zksync-era submodule
MarcosNicolau Aug 22, 2024
d0c59af
Add full tracers implementation
MarcosNicolau Aug 22, 2024
855d043
Refactor how the vm handles tracers
MarcosNicolau Aug 22, 2024
439f68e
Update era-compiler-tester submodule
MarcosNicolau Aug 22, 2024
e5ab604
Update zksync-era submodule
MarcosNicolau Aug 22, 2024
6d1682e
Address clippy warnings
MarcosNicolau Aug 22, 2024
0cc8684
Add default function in Tracer trait
MarcosNicolau Aug 23, 2024
081d615
Address clippy warnings
MarcosNicolau Aug 23, 2024
6068375
merge branch 'zksync-era-integration-tests'
juan518munoz Aug 26, 2024
ee97d09
Address review comments
MarcosNicolau Aug 26, 2024
5baf370
Add docs to state
MarcosNicolau Aug 26, 2024
75434eb
Address review comments
MarcosNicolau Aug 26, 2024
66d412c
Remove optional from run with tracers
MarcosNicolau Aug 26, 2024
291e4c1
Update era-compiler-tester submodule
MarcosNicolau Aug 26, 2024
02e8a90
Update era-compiler-tester submodule
MarcosNicolau Aug 26, 2024
ed3784c
Address clippy warnings
MarcosNicolau Aug 26, 2024
aea62fe
Fix storage_read
MarcosNicolau Aug 26, 2024
e6b1c19
Merge remote-tracking branch 'origin' into zksync-era-integration-tests
MarcosNicolau Aug 26, 2024
0385534
Merge branch 'zksync-era-tests' into implement-tracers
MarcosNicolau Aug 26, 2024
caad06d
Merge branch 'zksync-era-integration-tests' into implement-tracers
MarcosNicolau Aug 26, 2024
c06ce65
Merge branch 'main' into implement-tracers
MarcosNicolau Aug 28, 2024
f5dfd2a
merge branch 'main'
juan518munoz Aug 28, 2024
b444373
update zksync era submodule
juan518munoz Aug 28, 2024
fc2855a
Implement statistics (#213)
MarcosNicolau Aug 29, 2024
c36cf47
Merge branch 'main' into implement-tracers
MarcosNicolau Aug 29, 2024
5de6b4d
Update zksync-era submodule
MarcosNicolau Aug 29, 2024
adc3c0e
Move storage param to vm run
MarcosNicolau Aug 30, 2024
c1ab599
Merge branch 'main' into zk_server_compatibility
gianbelinche Sep 3, 2024
05c6bb9
Update zksync era submodule
gianbelinche Sep 3, 2024
beb8ef0
Update compiler tester submodule
gianbelinche Sep 3, 2024
264c0d0
Lint
gianbelinche Sep 3, 2024
5ba2e42
Merge branch 'zk_server_compatibility' into era_vm_rollbackable_priv_…
gianbelinche Sep 3, 2024
d9f581f
Remove print
gianbelinche Sep 3, 2024
1e6167a
Update zksync era submodule
gianbelinche Sep 3, 2024
2a213dc
Add heap size optimization
gianbelinche Sep 4, 2024
a44f3b8
Update submodules
gianbelinche Sep 4, 2024
c4dab0e
Address comments
gianbelinche Sep 4, 2024
80cf4c3
Merge branch 'zk_server_compatibility' into era_vm_rollbackable_priv_…
gianbelinche Sep 4, 2024
7f4d603
Merge branch 'main' into era_vm_rollbackable_priv_fields
gianbelinche Sep 4, 2024
274fa7e
Lint
gianbelinche Sep 4, 2024
9abd7fa
Update submodule
gianbelinche Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 52 additions & 4 deletions src/rollbacks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ pub trait Rollbackable {

#[derive(Debug, Default, Clone)]
pub struct RollbackableHashMap<K: Clone + Hash, V: Clone> {
pub map: HashMap<K, V>,
map: HashMap<K, V>,
}

impl<K: Clone + Hash + Eq, V: Clone> RollbackableHashMap<K, V> {
Expand All @@ -21,6 +21,18 @@ impl<K: Clone + Hash + Eq, V: Clone> RollbackableHashMap<K, V> {
}
}

pub fn insert(&mut self, key: K, value: V) {
self.map.insert(key, value);
}

pub fn get(&self, key: &K) -> Option<&V> {
self.map.get(key)
}

pub fn inner_ref(&self) -> &HashMap<K, V> {
fkrause98 marked this conversation as resolved.
Show resolved Hide resolved
&self.map
}

pub fn get_logs_after_snapshot(
&self,
snapshot: <RollbackableHashMap<K, V> as Rollbackable>::Snapshot,
Expand Down Expand Up @@ -55,7 +67,7 @@ impl<K: Clone + Hash, V: Clone> Iterator for RollbackableHashMap<K, V> {

#[derive(Debug, Default, Clone)]
pub struct RollbackableVec<T: Clone> {
pub entries: Vec<T>,
entries: Vec<T>,
}

impl<T: Clone> RollbackableVec<T> {
Expand All @@ -65,6 +77,14 @@ impl<T: Clone> RollbackableVec<T> {
}
}

pub fn push(&mut self, value: T) {
self.entries.push(value);
}

pub fn entries(&self) -> &[T] {
&self.entries
}

pub fn get_logs_after_snapshot(
&self,
snapshot: <RollbackableVec<T> as Rollbackable>::Snapshot,
Expand All @@ -89,7 +109,7 @@ impl<T: Clone> Rollbackable for RollbackableVec<T> {

#[derive(Debug, Default, Clone)]
pub struct RollbackablePrimitive<T: Copy> {
pub value: T,
value: T,
}

impl<T: Copy> Rollbackable for RollbackablePrimitive<T> {
Expand All @@ -103,9 +123,23 @@ impl<T: Copy> Rollbackable for RollbackablePrimitive<T> {
}
}

impl<T: Copy> RollbackablePrimitive<T> {
pub fn new(value: T) -> Self {
Self { value }
}

pub fn value(&self) -> T {
self.value
}

pub fn set(&mut self, value: T) {
self.value = value;
}
}

#[derive(Debug, Default, Clone)]
pub struct RollbackableHashSet<K: Clone> {
pub map: HashSet<K>,
map: HashSet<K>,
}

impl<K: Clone> Rollbackable for RollbackableHashSet<K> {
Expand All @@ -118,3 +152,17 @@ impl<K: Clone> Rollbackable for RollbackableHashSet<K> {
self.map.clone()
}
}

impl<K: Clone + Eq + Hash> RollbackableHashSet<K> {
pub fn insert(&mut self, value: K) -> bool {
self.map.insert(value)
}

pub fn contains(&self, value: &K) -> bool {
self.map.contains(value)
}

pub fn inner_ref(&self) -> &HashSet<K> {
&self.map
}
}
102 changes: 51 additions & 51 deletions src/state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,15 @@ impl VMState {
}

pub fn storage_changes(&self) -> &HashMap<StorageKey, U256> {
&self.storage_changes.map
self.storage_changes.inner_ref()
}

pub fn transient_storage(&self) -> &HashMap<StorageKey, U256> {
&self.transient_storage.map
self.transient_storage.inner_ref()
}

pub fn l2_to_l1_logs(&self) -> &Vec<L2ToL1Log> {
&self.l2_to_l1_logs.entries
pub fn l2_to_l1_logs(&self) -> &[L2ToL1Log] {
self.l2_to_l1_logs.entries()
}

pub fn get_l2_to_l1_logs_after_snapshot(
Expand All @@ -100,8 +100,8 @@ impl VMState {
self.l2_to_l1_logs.get_logs_after_snapshot(snapshot)
}

pub fn events(&self) -> &Vec<Event> {
&self.events.entries
pub fn events(&self) -> &[Event] {
self.events.entries()
}

pub fn get_events_after_snapshot(
Expand All @@ -111,28 +111,29 @@ impl VMState {
self.events.get_logs_after_snapshot(snapshot)
}

pub fn refunds(&self) -> &Vec<u32> {
&self.refunds.entries
pub fn refunds(&self) -> &[u32] {
self.refunds.entries()
}

pub fn pubdata_costs(&self) -> &Vec<i32> {
&self.pubdata_costs.entries
pub fn pubdata_costs(&self) -> &[i32] {
self.pubdata_costs.entries()
}

pub fn pubdata(&self) -> i32 {
self.pubdata.value
self.pubdata.value()
}

pub fn add_pubdata(&mut self, to_add: i32) {
self.pubdata.value += to_add;
let previous = self.pubdata.value();
self.pubdata.set(previous + to_add);
}

pub fn read_storage_slots(&self) -> &HashSet<StorageKey> {
&self.read_storage_slots.map
self.read_storage_slots.inner_ref()
}

pub fn written_storage_slots(&self) -> &HashSet<StorageKey> {
&self.written_storage_slots.map
self.written_storage_slots.inner_ref()
}

// reads shouldn't be mutable, we should consider change it to a non-mutable reference
Expand All @@ -142,16 +143,16 @@ impl VMState {
.storage_read_inner(&key, storage)
.map_or_else(U256::zero, |val| val);

let refund =
if storage.is_free_storage_slot(&key) || self.read_storage_slots.map.contains(&key) {
WARM_READ_REFUND
} else {
self.read_storage_slots.map.insert(key);
0
};
let refund = if storage.is_free_storage_slot(&key) || self.read_storage_slots.contains(&key)
{
WARM_READ_REFUND
} else {
self.read_storage_slots.insert(key);
0
};

self.pubdata_costs.entries.push(0);
self.refunds.entries.push(refund);
self.pubdata_costs.push(0);
self.refunds.push(refund);

(value, refund)
}
Expand All @@ -165,17 +166,16 @@ impl VMState {
.storage_read_inner(&key, storage)
.map_or_else(U256::zero, |val| val);

if !storage.is_free_storage_slot(&key) && !self.read_storage_slots.map.contains(&key) {
self.read_storage_slots.map.insert(key);
if !storage.is_free_storage_slot(&key) && !self.read_storage_slots.contains(&key) {
self.read_storage_slots.insert(key);
};

self.pubdata_costs.entries.push(0);

self.pubdata_costs.push(0);
value
}

fn storage_read_inner(&self, key: &StorageKey, storage: &mut dyn Storage) -> Option<U256> {
match self.storage_changes.map.get(key) {
match self.storage_changes.get(key) {
None => storage.storage_read(key),
value => value.copied(),
}
Expand All @@ -187,32 +187,32 @@ impl VMState {
value: U256,
storage: &mut dyn Storage,
) -> u32 {
self.storage_changes.map.insert(key, value);
self.storage_changes.insert(key, value);

if storage.is_free_storage_slot(&key) {
self.written_storage_slots.map.insert(key);
self.written_storage_slots.insert(key);
let refund = WARM_WRITE_REFUND;
self.refunds.entries.push(refund);
self.pubdata_costs.entries.push(0);
self.refunds.push(refund);
self.pubdata_costs.push(0);
return refund;
}

// after every write, we store the current cost paid
// on subsequent writes, we don't charge for what has already been paid
// but for the newer price, which if it is lower might end up in a refund
let current_cost = storage.cost_of_writing_storage(&key, value);
let prev_cost = *self.paid_changes.map.get(&key).unwrap_or(&0);
self.paid_changes.map.insert(key, current_cost);
let prev_cost = *self.paid_changes.get(&key).unwrap_or(&0);
self.paid_changes.insert(key, current_cost);

let refund = if self.written_storage_slots.map.contains(&key) {
let refund = if self.written_storage_slots.contains(&key) {
WARM_WRITE_REFUND
} else {
self.written_storage_slots.map.insert(key);
self.written_storage_slots.insert(key);

if self.read_storage_slots.map.contains(&key) {
if self.read_storage_slots.contains(&key) {
COLD_WRITE_AFTER_WARM_READ_REFUND
} else {
self.read_storage_slots.map.insert(key);
self.read_storage_slots.insert(key);
0
}
};
Expand All @@ -222,37 +222,38 @@ impl VMState {
// the slots to their final values.
// The only case where users may overpay is when a previous transaction ends up with a negative pubdata total.
let pubdata_cost = (current_cost as i32) - (prev_cost as i32);
self.pubdata.value += pubdata_cost;
self.refunds.entries.push(refund);
self.pubdata_costs.entries.push(pubdata_cost);
let previous_pubdata = self.pubdata.value();
self.pubdata.set(previous_pubdata + pubdata_cost);
self.refunds.push(refund);
self.pubdata_costs.push(pubdata_cost);

refund
}

pub fn transient_storage_read(&mut self, key: StorageKey) -> U256 {
self.pubdata_costs.entries.push(0);
self.pubdata_costs.push(0);
self.transient_storage
.map
.inner_ref()
MarcosNicolau marked this conversation as resolved.
Show resolved Hide resolved
.get(&key)
.copied()
.unwrap_or_default()
}

pub fn transient_storage_write(&mut self, key: StorageKey, value: U256) {
self.pubdata_costs.entries.push(0);
self.transient_storage.map.insert(key, value);
self.pubdata_costs.push(0);
self.transient_storage.insert(key, value);
}

pub(crate) fn clear_transient_storage(&mut self) {
self.transient_storage = RollbackableHashMap::default();
}

pub fn record_l2_to_l1_log(&mut self, msg: L2ToL1Log) {
self.l2_to_l1_logs.entries.push(msg);
self.l2_to_l1_logs.push(msg);
}

pub fn record_event(&mut self, event: Event) {
self.events.entries.push(event);
self.events.push(event);
}

/// Attempts to decommit the specified `hash` and retrieves any changes made since the initial storage state.
Expand All @@ -262,12 +263,12 @@ impl VMState {
/// - `Option<Vec<U256>>`: the contract bytecode
/// - `bool`: A boolean flag indicating whether the hash was decommitted (`true` if it was newly decommitted, `false` if it had already been decommitted).
pub fn decommit(&mut self, hash: U256, storage: &mut dyn Storage) -> (Option<Vec<U256>>, bool) {
let was_decommitted = !self.decommitted_hashes.map.insert(hash);
let was_decommitted = !self.decommitted_hashes.insert(hash);
(storage.decommit(hash), was_decommitted)
}

pub fn decommitted_hashes(&self) -> &HashSet<U256> {
&self.decommitted_hashes.map
self.decommitted_hashes.inner_ref()
}

/// Retrieves the values that have changed since the initial storage.
Expand All @@ -282,8 +283,7 @@ impl VMState {
&mut self,
storage: &mut dyn Storage,
) -> Vec<(StorageKey, Option<U256>, U256)> {
self.storage_changes
.map
self.storage_changes()
.iter()
.filter_map(|(key, value)| {
let initial_value = storage.storage_read(key);
Expand Down
Loading