Feat/l2 tezoracle

[AK-APRIORIT]

Summary

Adds direct native Pyth price reads to the existing Etherlink Michelson TezFinOracle through NAC staticcall_evm.

The production call path is:

Comptroller
  -> existing TezFinOracle.getValidatedPrice
  -> TezFinOracle.getPrice
  -> NAC staticcall_evm
  -> Pyth Core getPriceNoOlderThan(bytes32,uint256)

Changes

• Adds admin-configurable Pyth Core address, per-asset feed IDs, and Pyth freshness limit.
• Implements ABI calldata construction and decoding of Pyth Price:
  (int64 price, uint64 conf, int32 expo, uint256 publishTime).
• Normalizes native Pyth prices to TezFin target decimals.
• Enforces fail-closed validation for:
  • unavailable/reverted NAC calls;
  • malformed ABI responses;
  • non-positive prices;
  • unsupported exponents;
  • future timestamps;
  • confidence above the configured per-feed limit, checked as rawConf * 10,000 > rawPrice * maxConfidenceBps (BTC/USD: 25 bps, XTZ/USD: 50 bps, USDT/USD: 10 bps);
  • normalization overflow/zero result.
• Preserves existing getValidatedPrice checks and Comptroller-facing interface.
• Adds pinned mappings:
  • tzBTC -> BTC/USD;
  • XTZ -> XTZ/USD;
  • USDtz / USDt -> USDT/USD.
• Adds a shadownet deploy profile, staged oracle configuration script, read-only verification script, and smoke-test runner.
• Adds reproducible compiled contract/storage hashes and CI execution of offline Pyth ABI fixtures.
• The smoke script now compiles and redeploys from the same caller-selected directory. A new deploy_compiled_target.js uses the existing deployment flow, and the script checks the compiled artifact hash before redeployment
• Configuration validates the manifest chain ID against the connected chain before writes. The approval gate treats either a mainnet profile or a mainnet chain ID as mainnet; tests cover a mislabeled profile and a chain-ID mismatch.
• CI compares compiled-contract-hashes.json with the fresh reproducible build and fails if the tracked artifact is stale. The current file matches a fresh build
• Adds the on-chain measurement collector and the measure:pyth-confidence npm script, with a report of the completed 24-hour normal-period run

Pyth Configuration

Feed	Feed ID	Target decimals	Proposed confidence limit
BTC/USD	e62df6c8b4a85fe1a67db44dc12de5db330f7ac66b72dc658afedf0f4a415b43	8	25 bps
XTZ/USD	0affd4b8ad136a21d79bc82450a325ee12ff55a235abc242666e423b8bcffd03	6	50 bps
USDT/USD	2b89b9dc8fdf9f34709a5b106b472f0f39bb6ca9ce04b0fd7f2e971688e2e53b	6	10 bps

• Pyth Core: 0x2880aB155794e7179c9eE2e38200202908C17B43
• Pyth max age: 60s
• TezFin max price age: 60s

24-Hour On-Chain Measurement

Collected one sample per feed per minute from Etherlink Mainnet Pyth Core: 1,440 observations per feed. These are polling observations, not necessarily independent Pyth updates.

Feed	p50	p95	p99	Max	Rejections at proposed limit
BTC/USD	1.45 bps	2.36 bps	2.81 bps	4.83 bps	0; max used 19.3% of limit
XTZ/USD	2.92 bps	4.94 bps	5.72 bps	9.07 bps	0; max used 18.1% of limit
USDT/USD	0.77 bps	1.44 bps	1.80 bps	3.40 bps	0; max used 34.0% of limit

Unique publish_time counts were BTC 1,018, XTZ 1,020, and USDT 1,020. Confidence rejections added zero unavailability during this run.

Time-weighted system uptime, with all three feeds required to be fresh, was 38.908% at a 60s threshold, 92.813% at 180s, 98.457% at 300s, and 99.869% at 600s. The run recorded 16 stale episodes, a longest stale episode of 413 seconds, and 22 minutes 12 seconds of total stale downtime.

Validation

• npm test in deploy/deploy_script: 59/59 passed.
• bash contracts/tests/run_tests.sh ~/smartpy-cli/SmartPy.sh: 12/12 SmartPy suites passed.
• Pyth ABI fixture suite: 32 fixtures passed.
• test_operation_size.py: TezFinOracle 25,178 bytes; 7,590 bytes below the 32,768-byte limit.
• test_irm_wiring.py, test_deploy_pipeline_wiring.py, and test_mainnet_governance_payload.py: passed.
• compiled-contract-hashes.json matches a fresh clean build.

[KevinMehrabi]

I reviewed the current PR head (d13f633) and found two signed ABI decoding issues in contracts/TezFinOracle.py that should be fixed before approval:

1. _decodeExponentWord decodes the full 256-bit ABI word, then subtracts 2**32 for a negative exponent. A canonical exponent of -8 is encoded as 2**256 - 8; this calculation produces an enormous positive value rather than -8, so a normal Pyth quote fails INVALID_PYTH_EXPONENT. Please subtract 2**256 from the full decoded word, or first isolate the low 32 bits and then subtract 2**32. The canonical signed-int32 ranges must also be enforced.

2. _decodePriceWord treats every zero-extended word below 2**64 as a positive int64. Values from 2**63 through 2**64 - 1 have the int64 sign bit set and are not valid canonical positive int64 encodings. Please enforce the signed-int64 range and reject malformed sign extension.

Please add tests exercising the actual TezFinOracle contract decoder with a valid negative exponent and malformed signed words. The Python ABI fixture alone does not cover this contract behavior. After the fix, please rerun CI and the compiled-contract/hash checks and request re-review. This is separate from the acknowledged Shadownet/Pyth feed-testing work.

[AK-APRIORIT]

_decodeExponentWord now enforces the canonical signed-int32 range using the correct sign-bit boundary at 2**31. Negative values are converted from the full 256-bit two’s-complement word by subtracting 2**256, so canonical Pyth
exponents such as -8 decode correctly.

_decodePriceWord now enforces the canonical signed-int64 range using the 2**63 boundary. Zero-extended values from 2**63 through 2**64 - 1 are rejected as malformed instead of being interpreted as positive prices.

Contract-level SmartPy regression tests were added through a test-only harness that invokes the actual TezFinOracle decoder methods. The tests cover canonical negative exponents, malformed exponent sign extension, canonical positive and negative prices, and malformed signed-int64 values.

Validation results:

• 32 ABI fixtures and decoding checks passed
• full SmartPy suite: 12/12 passed
• operation-size check passed at 25,178 bytes, with a 7,590-byte margin
• reproducible compilation passed
• the tracked TezFinOracle compiled hash matches a fresh reproducible build

Updated TezFinOracle contract hash:

2af5e7c4e9b6b627782a338ec9eddcc52d3ed9a6c39cc33e791952bc014eb8f3

[KevinMehrabi]

I re-reviewed commit ebd1cf3269f67f6b06f6392a5c217f4529cce20d independently. The signed ABI decoder fixes are correct and the contract tests pass, but I found the following remaining blockers.

1. The Shadownet E2E script does not deploy the contract it compiles. e2e/shell_scripts/shadownet_pyth_smoke_test.sh:42-44 compiles TezFinOracle into /tmp/tezfin_oracle_compiled, but its REDEPLOY=1 path at lines 58-60 invokes deploy.js. That command uses TezFinBuild/compiled_contracts through util.js:525-529, not the fresh /tmp output. In a clean checkout that directory does not exist; in a reused checkout it may contain stale or unrelated full-protocol artifacts. Please make the smoke test originate the freshly compiled oracle artifact it just produced, using a fresh/copy-on-write Shadownet manifest, and add a regression or dry-run test proving the compile output consumed by the deployment step is the same output produced earlier in the script.

2. The “no mainnet override” confidence-limit guard trusts a profile label rather than the connected chain. configure_pyth_oracle.js:71 treats the run as mainnet only when config.networkProfile === 'mainnet', and it resolves the approval gate before createTezosClient() returns the actual chain ID. A configuration whose RPC and chainId point to mainnet but whose networkProfile is accidentally shadownet can therefore use ALLOW_UNAPPROVED_CONFIDENCE_LIMITS=1 on mainnet, contrary to the script's documented guarantee. Please derive this gate from the actual connected chain ID (and fail closed if either the profile or actual chain is mainnet), validate the deployment manifest's chainId against that connection before any writes, and add tests for a mislabeled mainnet profile and a mismatched manifest.

3. The checked-in compiled hash manifest is stale. Two clean reproducible builds, using Python 3.9 and Python 3.11, produced identical hashes to each other, but they do not match compiled-contract-hashes.json for CUSDt, CXTZ, Comptroller, CtzBTC, or Governance (including several storage hashes). Only the new TezFinOracle entry matches. CI currently overwrites this tracked path and uploads the newly generated file (.github/workflows/ci.yml:54-63) without checking whether the committed file was already current, so green CI does not catch the discrepancy. Please regenerate the entire checked-in manifest from a clean build and make CI fail when a fresh generated manifest differs from the tracked release artifact.

4. The documented confidence-measurement tool is absent and the required measurement is still incomplete. docs/PYTH_CONFIDENCE_MEASUREMENT_REPORT.md:16-51 documents deploy/deploy_script/measure_pyth_confidence.js, including collect-onchain, collect, and report modes, but that file is not present in this branch. The same report explicitly contains zero samples and marks every acceptance item incomplete at lines 70-99. Please commit the referenced collector and its tests/usage, then add the completed per-feed empirical output when the run finishes. Until that evidence is present, the proposed confidence limits must remain non-production and this PR should not be described as the frozen production audit handoff.

Please also update the PR description: it still describes the old 25% confidence rule, 13 fixtures, and the prior operation size, while this head uses per-feed bps limits, 32 fixture cases, and the smaller current artifact.

Independent verification completed on this head:

• 12/12 SmartPy suites passed under Node 22.16.0.
• All 32 Pyth ABI fixture cases passed.
• 36/36 deployment guard tests passed.
• IRM wiring, governance payload, deployment wiring, operation-size, and reproducible-build checks passed.
• The new canonical signed-int64/int32 decoder boundary tests passed.

Once the four blockers above are corrected, please request another review.

[AK-APRIORIT]

1. Shadownet E2E script now deploys the artifact it just compiled

e2e/shell_scripts/shadownet_pyth_smoke_test.sh previously compiled TezFinOracle into /tmp/tezfin_oracle_compiled, but its REDEPLOY=1 path invoked deploy.js, which deploys from TezFinBuild/compiled_contracts instead - a directory that may not exist, or may contain stale/unrelated artifacts.

• The script now uses a single oracle_compile_dir variable for both the compile step and the redeploy step, so they can never silently diverge.
• A new deploy/deploy_script/deploy_compiled_target.js deploys a caller-supplied compiled-contracts directory through the existing runDeployment(), instead of the hard-coded path in deploy.js.
• A dry-run guard computes a SHA-256 of the compiled artifact right after compiling and re-checks it immediately before redeploying, failing loudly if the artifact changed or is stale.

2. Mainnet confidence-limit gate now derives from the actual connected chain

configure_pyth_oracle.js previously trusted config.json's networkProfile label and resolved the approval gate before the chain id was known, so a mislabeled profile could bypass ALLOW_UNAPPROVED_CONFIDENCE_LIMITS on a real mainnet connection.

• createTezosClient() is now called first, and the manifest's chainId is validated against the actual connected chain before any write.
• The mainnet gate is now networkProfile === 'mainnet' || MAINNET_CHAIN_IDS.has(chainId), so either signal being mainnet blocks the bypass.
• Added unit tests for isActuallyMainnet and resolveApprovedConfidenceLimits, including a mislabeled-profile-on-mainnet-chain case and a manifest/chain mismatch case.

3. compiled-contract-hashes.json staleness is now caught by CI

A fresh clean build was regenerated and diffed against the committed file - it is already current for every contract (no changes needed). CI now adds an explicit git diff --exit-code -- compiled-contract-hashes.json step right after the reproducible-build step, so a future stale commit fails the build instead of green CI silently overwriting and re-uploading a mismatched file.

4. Confidence-measurement tool committed, with completed 24h evidence

deploy/deploy_script/measure_pyth_confidence.js (previously only documented, not committed to this branch) is now included, along with its measure:pyth-confidence npm script.

docs/PYTH_CONFIDENCE_MEASUREMENT_REPORT.md is updated with completed 24-hour on-chain results for BTC/USD, XTZ/USD, and USDT/USD (1,440 samples each, against Etherlink Mainnet Pyth Core):

• confidence ratios stay well below the proposed limits (max usage 19.3% for BTC/USD, 18.1% for XTZ/USD, 34.0% for USDT/USD; zero rejections in this run);
• per-feed observation counts vs. unique publish_time counts are reported separately, so polling frequency is not conflated with update frequency;
• time-weighted system availability is reported for 60/180/300/600s thresholds, with stale-episode counts and total downtime.

[KevinMehrabi]

Thank you for the update. I re-reviewed commit c1da35a8b837b4933b81e62062414d786c83012c. The mainnet-chain gate and compiled-hash corrections are resolved, and the new deployment guard tests pass. The following items remain before approval of PR #463:

1. REDEPLOY=1 still does not guarantee a fresh origination. The script now passes the newly compiled directory to deploy_compiled_target.js, which is an improvement. However, it still passes the existing Shadownet manifest, and that manifest already contains TezFinOracle. runDeployment() therefore verifies and skips the existing address when it matches, or rejects it when it differs; it does not originate the newly compiled oracle. Please use a fresh/copy-on-write manifest for the redeployment step, or explicitly remove only the copied manifest's TezFinOracle entry, and add a test proving that REDEPLOY=1 reaches a new origination without modifying the checked-in manifest.

2. The 24-hour measurement results are not independently reproducible from the PR. The report contains numerical results, but the underlying per-feed CSV observations are not included or attached. Please commit or attach the raw dataset (or an immutable, checksummed artifact) together with the exact collection period, RPC, Pyth Core address, and feed IDs so the reported figures can be independently regenerated.

3. The committed reporting tool cannot reproduce several figures in the report and has no tests. Its report mode currently calculates confidence percentiles and rejection counts only. It does not calculate the reported unique publish_time counts, per-threshold freshness, time-weighted system uptime, stale episodes, longest episode, or total downtime. Please make the tool deterministically generate every reported table/statistic from the supplied data and add focused tests for parsing, percentiles/boundaries, repeated publish times, freshness thresholds, downtime/episode accounting, and failed/missing feed observations.

4. The PR description still contains the obsolete 25% rule. Under “Changes,” it says excessive confidence is conf > price / 4, which conflicts with the feed-specific 25/50/10 bps policy later in the same description. Please remove or correct that statement.

The normal-period results are useful, but the report still marks stressed/high-volatility measurement as pending. That may remain an explicit production-activation gate rather than a code-merge blocker, provided the PR and audit handoff clearly state that the proposed confidence limits are not yet approved for production. Real-Pyth signed-update compatibility and any proxy-market activation likewise remain separate production gates.

Please request another review after the four PR changes above are addressed. PR #464 should remain review-only and must not be merged into the production branch.

[AK-APRIORIT]

1. REDEPLOY=1 now creates a separate manifest copy, removes only its TezFinOracle entry, and proceeds to a fresh origination. A test verifies origination occurs and the source manifest remains unchanged.
2. Added the three raw CSV datasets with 1,440 observations each, checksums, the exact collection period, RPC, Pyth Core address, and feed IDs.
3. The reporting tool now reproduces confidence statistics, unique publish_time counts, freshness, uptime, downtime, and stale episodes. Focused parser and reporting tests were added; npm test passes 59/59.
4. Prepared a replacement for the obsolete 25% rule in the PR description, using the per-feed limits: BTC 25 bps, XTZ 50 bps, and USDT 10 bps.

[KevinMehrabi]

Re-reviewed current head 4b444b0cfeb9f3e58f9f44c778816ef21703b403. The four remaining blockers from the previous review are resolved:

1. REDEPLOY=1 now uses a copy-on-write manifest, removes only the copied TezFinOracle entry, performs a fresh origination from the just-compiled artifact, and preserves the source manifest. The regression test exercises the origination path.
2. The three raw 1,440-observation CSV datasets are committed with checksums, collection window, RPC, Pyth Core address, and feed IDs.
3. The reporting tool now deterministically reproduces the confidence percentiles, rejection counts, unique publish-time counts, per-feed freshness, time-weighted uptime/downtime, and stale episodes. Focused tests cover parsing, boundaries, repeated quotes, failed/missing observations, and availability accounting.
4. The PR description now states the configured 25/50/10-bps feed-specific limits instead of the obsolete 25% rule.

Independent verification on this head:

• npm test in deploy/deploy_script: 59/59 passed.
• The committed raw datasets reproduce the published report and all three documented SHA-256 checksums exactly.
• GitHub contract, deployment-script, and TypeScript CI jobs are all green.
• The branch is mergeable and no new code-level blocker was found.

Approved for code merge. This approval does not authorize a production oracle switch or market activation. The stressed/high-volatility measurement, governance approval of the final confidence and freshness policies, reproducible real-Pyth signed-update compatibility on a healthy deployment, the production updater/monitoring plan, and separate depeg/activation policies for proxy-priced markets remain production gates. PR #464 remains review-only and must not be merged.