This is an RFC for authenticated JSON rpc API

Authentication

The engine JSON-RPC interface, exposed by EL and consumed by CL, needs to be authenticated. The authentication scheme chosen for thus purpose is JWT.

The type of attacks that this authentication scheme attempts to protect against are the following:

• RPC port exposed towards the internet, allowing attackers to exchange messages with EL engine api.
• RPC port exposed towards the browser, allowing malicious webpages to submit messages to the EL engine api.

The authentication scheme is not designed to

• Prevent attackers with capability to read ('sniff') network traffic from reading the traffic,
• Prevent attackers with capability to read ('sniff') network traffic from performing replay-attacks of earlier messages.

Authentication is performed as follows:

• For HTTP dialogue, each jsonrpc request is individually authenticated by supplying JWT token in the HTTP header.
• For a WebSocket dialogue, only the initial handshake is authenticated, after which the message dialogue proceeds without further use of JWT.
• For inproc, a.k.a raw ipc communication, no authentication is required, under the assumption that a process able to access ipc channels for the process, which usually means local file access, is already sufficiently permissioned that further authentication requirements do not add security.

JWT specifications

• The EL MUST support the at least the following alg HMAC + SHA256 (HS256)
• The EL MUST reject the alg none.

The HMAC algorithm means that symmetric encryption is used, thus several CL's will be able to use the same key, and, from an authentication perspective be able to impersonate each other. From a deployment perspective, it means that an EL does not need to be provisioned with individual keys for each caller.

Key distribution

The EL and CL clients MUST accept a cli/config parameter: jwt-secret, a 256 bit key, to be used for verifying/generating jwt tokens. If such a parameter is not given, the client SHOULD generate such a token, valid for the duration of the execution, and show the token in the output, which the user can then use to provision the counterpart client with.

Adds finalized value to the block tags enum.

Pre-Merge response

There are three options:

• Refer to a block that is likely to stay in the PoW canonical chain forever, i.e. a block which is e.g. 128 blocks behind the head
• Respond with error
• Respond with genesis block

I am slightly inclined towards the third option. It gives users a graceful option to check whether the Merge has happened or not, i.e. checking if finalized.number > earliest.number.

Whatever option we choose it needs to be specified somewhere. Probably in the description field to this enum.

Safe block

Another couple of values that are considered for addition to this enum is safe and unsafe. It worth noting that if these values and the corresponding functionality will be added after the Merge then unsafe tag will be used as an alias to the latest to avoid backwards incompatibility.

Description

I've been playing with the way of adding a "description" field to this enum in the openrpc playground. To be honest, it looks ugly as it seems there is no way to describe each value separately and there is also no way to include line breaks into the description. So, whenever we will have to utilize description it will be a sequence of sentences without breaks.

cc @timbeiko @lightclient @djrtwo @MicahZoltu @MariusVanDerWijden

Currently the CL is blind to the value of the locally built block and therefore, if configured with a builder, will always propose the remotely built block.

In order to allow the CL to intelligently decide between using the local or remote block, the EL could provide the value of its block in its response to engine_getPayload. The EL can be free to determine the value of its block as it sees fit (summing tx fees, checking feeRecipient balance pre/post execution, etc..).

The change will help to ensure validators propose the highest paying block in general, but more importantly will increase the cost of censorship by builders.

There has been a misunderstanding from the Nimbus team. Without specifying which client SHOULD generate the jwt token, it could be assumed that it's up to the CL client also. This modification request would suggest that only the EL client should generate the jwt token, not the CL client.

Link to discussion. (Nimbus discord)

Clients currently support an assortment of debug-like RPCs and commands. In an effort to simplify debugging chain disagreements and have more reusable debugging tooling across clients, I propose we standardize five basic JSON-RPC methods.

• debug_getRawHeader(hashOrNumber) -> bytes -- gets header RLP.
• debug_getRawBlock(hashOrNumber) -> bytes -- gets block RLP.
• debug_getRawTransaction(hash) -> bytes -- gets EIP-2718 encoded tx.
• debug_getRawReceipts(hashOrNumber) -> []bytes -- gets array of EIP-2718 encoded receipts.
• debug_getBadBlocks -> []BadBlock -- gets recent invalid blocks seen by the client.
• ~debug_setHead(number) -> null -- overrides the client and resets the head to a certain block number.~

edit: seems like the general consensus is to not expose setHead under debug.

Geth currently supports all of these (although the interfaces are slightly different at the moment). I see other clients support some of the methods.

I'd like to know:

1. Is this is a good minimal set of debug methods to standardize (open to add/removing methods as needed)?
2. How long would it take for clients to implement each of these if there were integration tests for them in hive?

Proposal

• Adds engine_getPayloadBodies method to the Engine API
• Given the block_hash values the method returns ExecutionPayloadBody objects which consist of transactions list of the corresponding payloads
• The logic of the new method maps on the logic of existing GetBlockBodies message in ETH protocol -- this is the crux of this proposal

Concerns

The main concern is that retrieval of the bodies by hashes doesn't allow for utilizing linearity of block data as if the request was made by block numbers. Utilizing the linearity property opens up an opportunity for more optimal way of accessing this information, especially it makes sense for the finalized part of the blockchain.

See https://github.com/ethereum/execution-apis/issues/137 and discussion thread for more details

Implementations

• Nethermind, GetPayloadBodiesV1Handler.cs#L40

This is an initial draft for versioning engine_getPayload to support the EL client's estimate of the value of the block.

This allows the CL to compare the value of locally built blocks to remotely build blocks received from relays/builders.

https://github.com/ethereum/execution-apis/issues/307

This PR is pending consensus on the optimal path to handle changes to the execution API.

Comments and/or improvements are welcome.

The spec does not adequately explain the difference between SYNCING and ACCEPTED for non-canonical branches.

For a block for a non-canonical branch to be ACCEPTED, it must be a "well-formed" chain of known ancestors. Such a block with unknown ancestors cannot be ACCEPTED and must instead be signaled as SYNCING.

The purpose of ACCEPTED is to signal that there is in fact a well-formed chain but it has not yet been executed. ACCEPTED is not intended to also be for non-canonical branches missing requisite data.

This specification uses EIP 1474, splits up the spec into a more manageable multi-file format, and I've begun incorporating the yellow paper where applicable.

This is the initial commit, and changes are expected. Particularly as they pertain to including edgecases as an md in each endpoints description. This will be accomplished by storing each edgecase in the same format as the methods themselves but in an "edgecases" folder.

the current structure allows for more easy reuse of components / schemas with method specific descriptions and requirements and it more clearly separates schemas from components from methods which are all different levels. This separation is necessary due to the way that OpenRPC parses the json.

You may notice that I use external references to internal entries in the json (i.e. ./File.json#/item instead of #/item). This is just a work around for a weird bug I ran into where internally referenced items would try to resolve relative to the externally imported component (i.e. file2 imports File.json#/item and then #/item is not in file2 so it errors).

you can see the built specification here

The goal here was maximum maintainability by keeping relevant info where it's most useful and never repeating things. I'd like to get rid of the components in the future fit this reason but they already to be a key part of the openrpc. Nevertheless I've done my best here to keep things as clean as possible. There are some ways to clean it up still. But I think it's fine for now. But what do you think?

Make explicit that when returning INVALID from payload validation, the child of the latestValidHash in the invalid payload branch, is indeed INVALID, allowing thus the consumer to prune any block in this branch starting from said child.

NOTE: Currently based on #83 and pointed to that PR as the base. Trying to keep the diff legible while continuing to be able to work. Will rebase and point this PR to main once #83 is merged

INVALID is insufficient to parse the validity of ancestor blocks that previously returned SYNCING. As per discussions at interop, add validAncestorHash in the return value to point to the latest (by block number) valid ancestor to the payload that is being processed. In the event of INVALID, this can thus invalidate an arbitrary number of blocks in the chain starting with the payload.

Did discuss this with @holiman to ensure that this is feasible in geth.

Additionally:

• Provide guidance on how long to try to resolve a payload's dependencies before responding with SYNCING -- SLOTS_PER_SECOND / 30 (0.4s)
• Do not make PoW vs PoS an exceptional case wrt SYNCING return value. Enforcing finding of dependencies of blocks with PoW ancestors rather than returning SYNCING could result in deadlocks.

Introduces Engine API specification of EIP-6110.

Note: Methods and structures modified by the EIP are versioned as V6110, while structures that are added by the EIP are versioned as V1. This is done to avoid potential clashes when several experimental features and hard forks are modifying the same structures and methods.

An alternative to https://github.com/ethereum/execution-apis/pull/320. This PR deprecates engine_exchangeTransitionConfiguration by marking the method as deprecated rather than making a change into its specification.

There are two ways to deprecate the call on Mainnet:

1. As a part of Capella/Shanghai deployment process. Shanghai compliant EL clients don't expect this call to be made (may be entirely removed) and Capella compliant CL clients stop making this call. There could be a hiccup in this process when a user upgrades one layer but leave the other one not upgraded yet and start seeing worrisome messages in logs.
2. In an uncoordinated fashion. CL keeps calling this method but adds a check that this method does exist, and if EL responds with this method doesn't exist message CL doesn't make any noise in logs. This allows for EL to remove the method support, and once every EL client removes it then CL is free to do the same.

The first option looks simpler to me but I am fine either way.

This PR proposes the following cleanups to the Shanghai part of the spec:

• Sets timeouts whenever missed
• Makes validation of terminal PoW block conditions optional, so, client devs may remove it if they want
• Requires EL clients to stop using INVALID_BLOCK_HASH status and supplant it with INVALID, see https://github.com/ethereum/execution-apis/issues/270 for details (should be a one liner, if not let's debate)

If EL client implementations use the same piece of code for both V1 and V2 the latter two changes may become a problem as they seem to be backwards incompatible with V1. Though, practically it isn't that true as the transition has happened quite long time ago and having no TTD validation should not break anything except for Hive tests (though we should probably remove transition tests from Hive). All CL clients handle status: INVALID, latestValidHash: null in the same way as status: INVALID_BLOCK_HASH, latestValidHash: null, replacing the latter with the former must not change anything.

Unifies return value for the case when the wrong data structure version is used in V2 method calls in a way that client software MUST return -32602: Invalid params error.

The idea behind this change is to make a future proof unification to this type of failure, and also get rid of abusing INVALID payload status and discrepancy between two methods in handling the same type of error that exists in the current version of the spec.

Note: We may introduce a new custom error type to be returned in the response. I just thought we might not need it but not opposed to it in general.

When reorg happens, CL or EL clients take the reorg'ed objects (i.e. txs, attestations, exits... etc) and put them on the new canonical chain, so they are not lost. EL client handles txs reorging, but by itself, it's not sufficient to handle blob txs, because the blobs are not included within the blocks. EL clients may need more visibility to see all the blobs, such as flash bots inclusion. To satisfy this, CL needs to support EL. Here are a few options with tradeoffs:

1. CL pushes blobs to EL on every newPayload call.

2. CL pushes reorg'ed blobs hashes to EL on every forkchoiceUpdated call. EL can request them via p2p

1 and 2 are less ideal from our perspective. It couples EL and CL in ways we aren't comfortable with and in deeper ways than we had envisioned. CL clients will be required to make decisions about execution and deal with extra complexity. 1 also increases the latency, which delays state transition. It's probably best not to couple this resolution closing with engine API. Instead, the CL client could optionally call sendRawTransaction for reorg'ed blob txs, and the blob users could do the same too. It's also probably safe to assume blob users are more sophisticated, and they can also monitor blob txs status and resubmit in the event of reorg. Here is option 3

3. CL optionally can call sendRawTransaction on reorg'ed blob tx