New features

1. sim Explorer now supports Base 🥳. Check out USDC on Base. Anything you can do with an Ethereum address, you can now also do with a Base address. The Explorer home page is also now multichain.

   1. Note that we've had Base in sim Studio for ages. See some data you like in Explorer and want to build it in Studio--reach out!

2. Test EVM Lambda or Patch components on a specific transaction instead of a block range. We were prototyping a traces core table recently (more on this soon!) and running the test interaction even on a single block gave us way more traces than we needed/wanted to spot check. Now you can just give a single transaction hash and test on that, if you prefer.

3. 

   Emitting data directly to Postgres: With the schema mirror, you can mirror sim data to an external Kafka topic and then ingest from there into your preferred DB. If you want to send it to Postgres, you can now skip the Kafka middleman!

   

4. FOIA -> sim Explorer: On EoA pages on Explorer, we have our new Freedom of Information Act (FOIA) component that shows all instances of the given address in mappings in contracts' storage. We also have a dedicated route for just this data at https://explorer.sim.io/foia.

   

5. Playground dashboards: You may have noticed that we're including some simple dashboards like this dex bot dashboard and this trader PnL one in our X threads. These were created by non-engineer team-members with some AI assistance and we decided to share them at playground.sim.io. Turns out it's not that hard to build a useful FE if you have the right data from sim Studio. Shh! Don't tell our FE engineers I said that!

6. Improved mechanics for query results and lambda/patch logs panes: If you ever had issues with pane sizing within the Query Editor or IDE, e.g., your SQL got stuck underneath the query results, please accept our sincere apologies. Our custom logic turned out to be less than ideal. We're now using a library built for this. If you see something wrong, let us know.

New features

1. Component tables in org dashboard: Sometimes you might find yourself looking for a particular API, Persistence, or Webhook but you don't remember what canvas it's in. Or you might want to see all of your components of a given type in one place. Now you can for APIs, Persistences, and Webhooks in the org dashboard. Clicking on any row opens a component detail page.

2. Copy ABI button in sim Explorer: We added a button so you can copy the raw ABI from a contract page on Explorer to use it for an ABI hook or other purposes (e.g., modifying the ABI of a beacon proxy to that of its implementation to support an address hook).

3. Bribes coverage on sim Explorer: We added coverage of builder bribes to sim Explorer. You can see bribes for each transaction in block pages (e.g.) and we have a standalone bribes leaderboard at explorer.sim.io/bribes.

4. LP deployment and liquidations core tables: Check them out at LP deployments and Liquidations.

5. Improvements to "Add component from edge" flow: You can now add new components or connect existing ones (via selection or dragging) by clicking the node edge. And we added icons.

   

Coming soon

1. Improved mechanics for query results and lambda logs panes: Currently we have a few minor bugs involving the sizing of panes in our Query Editor, Patch, and Lambdas. For instance, in the query editor, some of your SQL gets hidden under the query results pane and you can't properly scroll to it. You may also have noticed that the sizing gets messy when you drag from one window to another. We have a much more robust experience, with fixes for these bugs, coming soon.
2. Emitting data directly to external DBs: With the current schema mirror, you can mirror sim data to an external Kafka topic and then ingest from there into your preferred DB. For certain DBs, starting with Postgres, we're working to make it even easier to directly ingest the data from sim in to the DB, without needing an external Kafka topic.
3. Base support for sim Explorer: It's gonna be so based!
4. FOIA -> sim Explorer: We'll add a new page to sim Explorer (and a new component to EoA pages) relating to our existing Freedom of Information Act (FOIA) query template, which allows you to see all instances of an address in contract mappings.

New features

1. Clearing schemas: This is our most requested feature! Before, if you had any running executions and you wanted to make a change to your schema, you had to duplicate the schemas and persistences, delete the originals, reconnect the duplicates and recreate the execution edges. Now it's much easier: simply use the Clear interaction in the schema component's menu. It clears/resets all downstream components (including persistences) and upstream execution edges, and puts the schema back into an editable state, ready for your edits. Check out the video:

2. Clearing persistences: If you just want to clear a persistence without touching the schema, simply pause any connected executions and use the Clear interaction in the persistence's menu. This can be handy if you don't need to edit the schema or lambda but want to remove all existing data from the table (e.g., to avoid duplicates) before running on a new block range.

3. Table name editing with conflict resolution: Want to build a new version of your table without causing downtime for connected APIs? Simply create the new table under a new name. Then, when it's ready for production, rename it to the name of the existing table. The new conflict resolution flow will allow you to claim the name for the new table while also renaming the current table holding that name.

4. Custom type support: When you emit a variable from sim Lambdas or Patches, you typically pick from one of the standard available types: address, bool, bytes, bytes32, int256, int64, string, uint256, uint64. If you connect a persistence, we automatically ingest these into the table using DB types that are best suited for each. Sometimes, however, you may want to emit data that doesn't fit neatly within these standard types. Now you can define your own type in a Schema types tab that's global to the canvas and use that type in the schema. Read more here.

5. Cross-lambda shared code: Ever found yourself declaring the same util functions in ten different lambdas of the same canvas. Then you have an idea to improve it and you have to update ten places? We now have a Utility tab in the Lambda/Patch IDE that you can use to define functions. They're automatically shared across all Lambdas/Patches within the contract.

Coming soon

1. Emitting data directly to external DBs: With the current schema mirror, you can mirror sim data to an external Kafka topic and then ingest from there into your preferred DB. For certain DBs, starting with Postgres, we're working to make it even easier to directly ingest the data from sim in to the DB, without needing an external Kafka topic.
2. Component tables in org dashboard: Canvases are great, but sometimes you want to see and manage all of your org's APIs, Persistences, and Webhooks in one place. We're adding these views to the org dashboard.
3. Canvas labels and descriptions: Soon you'll be able to add labels to your canvases to organize them better into projects or simply designate some as Production. We're also working on enabling you to give descriptions to canvases.
4. New sim-powered features for Explorer: sim Explorer is built on the infrastructure we use to power sim Studio. More and more, we're using Studio canvases ourselves to add new features to Explorer. In our immediate plans, we're using Studio to add coverage of builder bribes in Explorer. We'll also add a new page relating to our existing Freedom of Information Act (FOIA) query template, which allows you to see all instances of an address in contract mappings.
5. Improved mechanics for query results and lambda logs panes: Currently we have a few minor bugs involving the sizing of panes in our Query Editor, Patch, and Lambdas. For instance, in the query editor, some of your SQL gets hidden under the query results pane and you can't properly scroll to it. You may also have noticed that the sizing gets messy when you drag from one window to another. We have a much more robust experience, with fixes for these bugs, coming soon.

New features

1. Public canvases and shared queries: Want to share your canvas or a query with someone outside your org? Perhaps they haven't even signed up for sim.io? Just give them a canvas URL or use the share query feature. Anyone can open any canvas link and view its contents in read-only. Anyone can open a shared query link and run that query. The ability to mark a canvas as private, restricting access to org members, is coming soon for our paid tiers.

2. Add new components from handles of existing components: Adding all the new components you need to your canvas using the left sidebar is so yesterday... Instead, click on either handle of any existing component and you'll see a menu of the components that can be connected downstream (or upstream, for the left handle) of the existing component. Click your choice and it's added and automatically connected to the existing node.

   

3. Format SQL: Your SQL is perfect, but your colleague... well... it's not great... Fortunately, there's now a Format SQL feature within the hamburger menu of the query editor.

   

4. New API templates and core tables

   1. Uniswap V3 on-chain prices (X thread)
      1. getPriceFromUniswapV3 and @sim.uniswap_v3_prices
   2. Basenames data (related X threads: 1, 2, 3)
      1. basenames_expiries
      2. basenames_payments
      3. basenames_primary_names
      4. basenames_texts
      5. basenames_reserved
   3. contract_deployments

Coming soon

We're tantalizingly close to solving some major annoyances for users:

1. Table name editing with conflict resolution: You realized there was something you want to improve in your old table but you're using it in production.
2. Clearing schemas and persistences: You wrote some bad/incomplete data into a schema and persistence. There's no shame. We've all done it. But to reset the board, as punishment for your sins, you're going to have to duplicate the existing schema and persistence components as well as the execution edge. Then you delete the originals, reconnect the duplicates, and restart the executions. Don't forget to fix the names! It sucks, especially on bigger canvases with many components flowing into the same persistences. We've decided that the punishment doesn't fit the crime. Instead, we'll give you an option to simply clear the existing schemas and persistences.
3. Complex type support: Sometimes you might want to emit a struct to a schema and have it show up as a JSON in your table. Soon you'll be able to!
4. Cross-lambda shared code: Ever found yourself declaring the same util functions in ten different lambdas of the same canvas. Then you have an idea to improve it and you have to update ten places? Solution coming soon.

New features

1. We have a raft of changes coming to make it easier to share your work and view that of others. The first has already landed: When you're signed in to your org and viewing a canvas from another org, you previously couldn't run test cURLs against the API components because the API key was (correctly) shrouded. Now, however, we're allowing you to test APIs in other canvases with an API key that's specific to your org and works for all API components. This way, you can test other APIs without impacting their org's usage. It's automatic: just use the Copy cURL request feature on an out-of-org canvas and we'll provide the cURL with your org's key. See the coming soon section below for more related changes.
2. Table name editing: A while ago we created table name abstractions / aliases for internal purposes so that we could update and rebuild core tables like @sim.erc20_balance_changes without any downtime for users. Of course, we've heard from you that you want the ability to do the same for your tables! And now you can: simply edit the table name within the persistence component. For now, if you have two tables table1 and table2 and you want to rename table2->table1, you'll have to first rename table1-> any other name to free up the table1 name. Coming soon (see below): a way for you to do both changes in one fell swoop, resulting in no downtime at all.
3. More / better core tables and new API templates
   1. @sim.dex_trades: Normalized DEX trades on Ethereum (Uniswap V2/3, Balancer V2, PancakeSwap V3, Curve.fi) and Base (Uniswap V2/3, Balancer V2, Aerodrome, Infusion, Maverick V1, BaseX, PancakeSwap V3, CrocSwap/Ambient).
   2. @sim.mapping_storage_writes: Changes in values in Solidity mappings in contracts. Includes the real key (hash preimage). Includes Ethereum, Base, and Arbitrum.
   3. @sim.transactions: Summaries of transactions on Ethereum, Base, and Arbitrum.
   4. @sim.blocks: Block summaries. Currently Ethereum only.
   5. getFoia template: Inspired by the Freedom of Information Act, returns all instances in which a user-specified address is included as a key in a mapping.
4. Canvas metadata: Click on the icon at the top of any cavas to see some metadata about it.

5. Patch contract navigation: When you target a contract with the Patch component, we flatten all of the source code into a single file. On most contracts, you can now also navigate to any function or log where you want to include your patch using the Contract navigation feature in the left sidebar. Often it's more efficient that CTRL/CMD-F.

   

Coming soon

Lions and tigers and bears, oh my!

1. If you're not signed in and you visit a canvas, unless it's one of a few of our demo canvases, you're prompted to sign-in before you can view the canvas. Soon anyone will be able to open and view any canvas in read-only mode without signing in. You'll even be able to run test executions on lambdas and test queries in API components. Don't worry--you can't alter the state of someone else's canvas! As always, if you like a canvas, you can sign in, duplicate it into your org ,and modify as you wish. We'll also have more improvements coming to the non-signed-in experience soon.
2. Public org pages: want to know what we're up to in the sim org? All of our canvases our publicly accessible but right now, unless you're in the sim org, you can't see our org dashboard. Soon we'll have public org pages so that you can visit studio.sim.io/sim to see all our org dashboard, regardless of whether you're signed in and what orgs you belong to.
   1. When we introduce paid plans, one feature will be the ability to mark a canvas or an entire org as private so that they're only visible to users within the org. In the meantime, if there's anything you want private, reach out to us on Telegram and we'll do it for you manually (for free 😃).
3. Clearing schemas and persistences: You wrote some bad/incomplete data into a schema and persistence. There's no shame. We've all done it. But to reset the board, as punishment for your sins, you're going to have to duplicate the existing schema and persistence components as well as the execution edge. Then you delete the originals, reconnect the duplicates, and restart the executions. Don't forget to fix the names! It sucks, especially on bigger canvases with many components flowing into the same persistences. We've decided that the punishment doesn't fit the crime. Instead, we'll give you an option to simply clear the existing schemas and persistences.
4. Emit complex types: Sometimes you might want to emit a struct to a schema and have it show up as a JSON in your table. Right now you'd have to emit each member of the struct separately, but soon we'll allow you to emit structs and other complex types directly if you so choose.
5. Table name editing with conflict resolution: I mentioned above that if you want to rename a table with a name that's already taken, you first have to change the name of the conflicting table to free up the name. Soon you'll be able to do these together.

We're launching!

We're thrilled to announce that www.sim.io will be launching publicly tomorrow at 1 p.m. ET, following months of alpha testing and improvements. Your invaluable insights and support have been crucial in getting us here. We're immensely grateful for your time and expertise, especially as we didn’t know many of you before.

We'd greatly appreciate your help once more by asking you to share/quote our announcement Tweet/blogpost on X/Farcaster when it’s posted tomorrow. Your support could significantly boost our reach!

sim has been an incredibly ambitious project, especially for our small team. We've taken some bold steps that set us apart from the current offering and unlock new functionality:

• Our own EVM client optimized for instrumentation and parallel historical execution — at the current state it is already multiple orders of magnitude faster in historical execution than the fastest clients, letting users re-execute entire chains quickly.
• Unparalleled ability to hook, track and decode storage operations, letting you hook, track and understand storage/state changes.
• First EVM (that we know of) that can operate in a cluster and execute different chains simultaneously, allowing us to scale up, support faster backfills for more customers with less hardware.
• Introduction of our EVM Lambda, a serverless EVM Lambda enabling you to seamless interact with on-chain state in an off-chain computation enviorment (currently these off-chain computations are written in solidity, but our goal is to enable any language that compiles down to WASM)
• Bespoke streaming, compute and storage infrastructure that lets you built truly real-time applications
  -Visual IDE and workflows which lets you interact with the most low-level primitives with ease (With time we do intend to support a CLI and declarative code experience in the future)

These innovations allow sim to offer unparalleled capabilities for applications and pipelines that rely on onchain data.In the coming days and weeks, we'll be sharing various examples and tutorials to showcase sim's capabilities.

Thank you again for your continued support. We're excited to see what people will build with sim and how it might transform the ecosystem!

Stateful lambdas

By default, both EVM Lambda and Patch components have their state reset at each block. The benefit of this is it allows for very fast backfills as we can parallelize block execution. In some cases, however, it's really useful to maintain some state across blocks and you only want to run at the tip (or you're patient enough to tolerate sequential backfill). While it's still a very new feature that we're testing and improving, you can now run a stateful lambda. Learn how in our docs. If you're giving it a go, we'd also love to hear from you on Telegram as we're interested in learning more about your use cases.

Canvas locking/unlocking

Most of our sim.io orgs are active construction sites where we're supporting production apps in some canvases and prototyping new ideas in other canvases. If you're not careful, you could accidentally modify an important canvas. To reduce this risk, we've added a canvas locking and unlocking feature in the canvas. You can unlock or lock a canvas from the orgs dashboard or within a canvas (in the top menu). When a canvas is locked, it will render for all users, even those in the org, as a read-only canvas to avoid accidental edits. Want to make an edit? Just unlock it, make your edit, and lock it once more.

New features

1. Patched contracts: You can now customize blockchain executions by directly modifying an existing contract with the Patch component. Patch complements our existing EVM Lambda component, in which you define the code you want to execute on-chain as well as hooks that determine when that code executes. With a patched contract, whenever execution would have involved the contract that you patched, we swap in your patched contract instead. For many purposes, from emitting a little extra data to performing complex simulations, either component can be used. For both, state is reset at the end of the block (more on that below!) to enable parallelization for lightning-fast backfills. So which should you use?

   1. Lambdas are more flexible: You can insert execution across many contracts (e.g., that share an ABI), or upon each transaction, or each storage write, or each block, etc. None of these is possible with Patch.
   2. Patch puts you within the contract: If you're interested in a single contract, and particularly if you'd like to emit transient values from it, Patch is very powerful.

   When you choose which contract you want to patch, we automatically flatten all of the source code, similarly to the evm.codes contract viewer. As you make edits, toggle on Show diff to see the diff between the patched contract and the original. Test in the IDE and deploy a backfill via connecting to a data source, just like the Lambda. For more on Patch, check out our docs and try one yourself.

2. Multichain tables/APIs: Our core tables and API templates are now multichain. You'll see chain_id as a column in most tables. For the API templates, it's often an optional parameter--if you want to execute the requests across chains, no worries--just leave the default chain_id = 0 and we won't filter on chain.

Coming soon

1. Stateful probes (as discussed last changelog)
2. alpha -> beta + public launch (as discussed last changelog)

New features

1. Auto-generated schemas: When you're adding a hook in a lambda, you can choose to automatically generate and populate a corresponding schema. This means your lambdas will be fully testable, and in some cases ready to be deployed for backfills, without you having to write any Solidity at all.

2. Schema mirrors: You can now mirror data from a sim schema component into your own Generic Kafka or Confluent Cloud topic. Create the destination topic first, then create a new Schema mirror component in you canvas, connect it to an existing schema component, and point it at the destination topic.

3. API templates: We've been building core tables to make it easier for you to serve common query/API patterns without having to build the data yourself in a lambda. We're now going further in this direction by providing API templates on top of them. When defining an API query, explore the templates via the sidebar, run them, and choose Use this template as API to insert a template into the API Query tab. If it's exactly what you want, great. If not, customize it to your specific requirements.

4. Block filters for faster backfill: Sometimes when you do a backfill execution for your lambda, there are very few blocks within the block range in which the hooks actually intercept executions. Previously we'd still execute all blocks in the block range despite this, but now we have filters so that we'll only execute blocks relevant to your lambda. As a result, backfills are much faster, especially on sparser hooks. We're still improving how the filters work for more active contracts.

Improvements and bug fixes

1. You can now manually define the API key for API components, both at initial setup and in subsequent edits.
2. Query params used to reset whenever the query was changed at all, and when execution of the query failed. We've made them much stickier.

Coming soon

1. Open alpha: We're preparing to remove the waitlist and move to an open alpha. We'll be limiting certain features for new users. As part of this effort, we're also beefing up our execution and DB infra, which should result in a more performant and consistent experience for existing users.
2. Stateful lambdas: Currently any state you generate within a lambda gets reset every block. This is usually very nice as it means we can safely parallelize backfills, and aggregations can be done downstream or at query time. For some use cases, however, calculating and adjusting aggregations across blocks within the lambda’s state is very powerful, so we're launching a stateful version of the EVM lambda in the near future. These may be more costly to run and won’t backfill as fast, but they're extremely powerful for specific use cases.
3. A new patched contract component (as mentioned last changelog).

New features

1. When you add an address hook, you can now manually define and edit the associated ABI. This is handy for cases where we don't have the ABI (e.g., it's unverified) but you do, as well as when you want to edit/override the ABI that we have for the inputted address.

2. When you add an interface, you can also now input an ABI directly. You can input an entire ABI or just the functions you need. Interfaces can also always be renamed and edited in the Interface code tab.

3. We now support Base Sepolia. We don't fetch Sepolia contracts, so it's a BYOABI (bring-your-own ABI) situation for address hooks. Aside from that caveat, all hook types should work as they do on other chains.

Coming soon

1. A productized approach to schema mirroring, allowing you to ingest sim data into your own DBs. We're already doing this manually for some users, so reach out if you'd benefit from it.
2. API templates. We want to make it easy for everyone to replicate third-party endpoints they already use directly in sim. Want to get an account's NFT portfolio or see a token's holder leaderboard? No need to start from scratch, as we'll have in-product templates ready to use/edit/augment.
3. A new Patch component. Some uses cases (e.g., extracting transient values) can be hard to do with the EVM lambda. To better handle these, we're adding a new component that allows you to run a patched version of a contract. Within the patched contract, you're still able to execute custom logic and emit data to schemas, just as you do in the EVM lambda. We recommend the EVM lambda whenever possible as it's more flexible, capable and extensible, but the new component will be helpful for some use cases.

Bug fixes

1. In the canvas, we were showing incorrect addresses for address hooks in the EVM lambda component. This is now fixed.
2. Adding an interface sometimes introduced compiler errors. We believe we've now fixed this universally.
3. The query editor was showing all org queries instead of just your own. No longer.

EVM lambda redesign

We've rebuilt the EVM lambda component to make it both simpler and more functional. It includes:

1. A unified sidebar to manage hooks, schemas, and interfaces.
2. The ability to connect and edit an existing schema or create a new one without leaving the component.
3. A more linear flow from hook definition through to testing, combined with richer inline documentation.

Here's a walk-through of the changes:

{`

Bug fixes

1. Fixed bug where transaction hooks against different addresses within the same lambda would overwrite one another.
2. Fixed bug where you could only add evm.storage variable hooks if you also hooked on the contract of the variable.

Coming soon

1. A feature that allows you to manually input the ABI of an unverified contract, or a verified one that we haven't yet fetched, to enable function and log hooks.
2. We've heard from users that support for Base Sepolia would be helpful. Working on it. Initial support will be limited, won't include contract fetching.
3. A productized approach to schema mirroring, allowing you to ingest sim data into your own DBs. We're already doing this manually for some users, so reach out if you'd benefit from it.