, , and off-chain are advancements within cryptocurrency that use alternative methods to achieve faster , latency, and more. When one searches for benchmarks related to these technologies, the internet hands over a pile of extremely impressive figures. One architecture claims thousands of transactions per second. Another claims near-instant . A third claims fees measured in fractions of a cent. If we put those numbers side by side, it looks like a straightforward comparison. This is not exactly the case, however, and that's what this blog post aims to dive into.
Almost none of the benchmarks mentioned earlier were measured the same way, against the same conditions, using the same definitions of what counts as a transaction, a cost, or a finished payment. Comparing them directly is closer to comparing a car's top speed on a closed track to another car's average speed in city traffic, which leads to a very hazy view of what each technology's advantages and disadvantages are.
Bar length is logarithmic: every tenfold rise is the same distance, because 15 and 65,000 cannot share a linear axis. None of these figures states its workload, hardware, validator count or definition of a transaction, and several are theoretical maxima rather than observed throughput. † Terra stopped operating in 2022. This is the pile of numbers the rest of the post is about, not a ranking the lab stands behind.
The Industry Doesn't Even Agree on What "TPS" Means
Let's start by looking at the metric everyone reaches for first: transactions per second. This is something that one would assume is standardized, but this is not exactly the case. Independent analysis of blockchain benchmarking has pointed out that most research studies that test these tools don't do so under the same workload and the same environment. One chain advertised tens of thousands of TPS, while another advertised very little TPS. The work surrounding these technologies can reflect completely different definitions of "transaction," among other features that are not standardized. A simple payment on one network compared to a complex smart-contract call on another are not interchangeable, so this begs the question, how can we compare existing metrics of just one technology, without even branching out to compare multiple technologies against each other?
Independent research on blockchain performance measurement has made the same point from a different angle: not all transactions are equal, and without breaking performance down into multiple components instead of one headline number, comparisons end up obscuring what matters. A payment channel processing a simple balance update and a sharded network processing a cross-shard smart-contract call are not doing the same amount of work, even if both get reported as "one transaction."
What gets counted as a single transaction?
Sharding
A cross-shard contract call
Coordination between shards before it can be committed.
Sidechain
A smart-contract call on its own chain
Its own consensus, its own gas, its own validator set.
Payment channel
A balance update between two parties
No consensus round at all while the channel stays open.
All three get reported as the number one. That is the first reason a TPS figure from one architecture cannot be set beside a TPS figure from another.
Short Bursts Aren't the Same as Sustained Load
Even when two systems are measuring the same kind of transaction, the conditions under which that number was produced matter enormously. Analysis from the team behind Celestia has argued that the primary reason blockchain benchmarks mislead is that they run short bursts of traffic rather than sustained load. There's very clearly a large difference between a system that can accept a huge volume of transactions for a few minutes and one that can sustain that volume indefinitely in production, with all the storage growth, indexing overhead, and syncing demands that come with continuous operation, 24/7. A number pulled from a ten-minute test on optimized hardware does not speak for the technology as a whole.
This isn't a hypothetical concern for sharded networks specifically. Zilliqa, one of the earliest public blockchains to implement sharding in production, published testnet results in 2017 showing 2,488 transactions per second using six shards and 3,600 nodes, a test run entirely on Amazon Web Service's Singapore data center, using up the region's available server capacity in the process. While that number is real, the methodology has a few holes in it that are worth digging into. Their tests have nodes sitting close together in one cloud region, with no real-world latency between continents, no adversarial nodes, no competing background traffic, and no sustained multi-day operation. It describes what sharding can do under laboratory conditions, not necessarily what a geographically distributed, publicly reachable production network handling real traffic would sustain.
2,488 TPS
Zilliqa testnet, 2017. 6 shards, 3,600 nodes, a single AWS region, Singapore.
Conditions the test did not face
- Nodes sitting close together in one cloud region
- No real-world latency between continents
- No adversarial nodes
- No competing background traffic
- No sustained multi-day operation
The number is real. It describes what sharding did in that test, which is not the same claim as what it would sustain in production.
Hardware, Validator Count, and Geography Are Rarely Held Constant
A sharded network's throughput scales with the number of shards and the number of available to fill them. A sidechain's throughput depends on how many validators it runs and how demanding its consensus mechanism is. A payment channel network's effective throughput depends on how much liquidity is actually available and how well-connected the channel topology is. None of these numbers are architecture-neutral, meaning that they all rely on different architectural capacities to increase their throughput. This makes it extremely hard to develop a fair comparison because having a low number of validators, for example, could negatively impact a sidechain while not having an impact on an off-chain technology.
It is also important to consider what changes between a benchmark run on a handful of validators in a single data center versus one run across globally distributed nodes with realistic network latency and packet loss between them. Real-world guidance on designing fair blockchain benchmarks emphasizes exactly this; results need to be objective and verifiable by anyone through open scripts, configurations, and disclosed machine specs. If this is not possible, a benchmark's headline number can't be trusted to represent anything beyond the specific hardware and network topology it was run on. A sidechain benchmarked on ten validators in one region and another benchmarked on a hundred validators spread across five continents are not comparable numbers, even if both get reported simply as "TPS."
Hardware
Machine specs are often undisclosed, so the number belongs to the machine.
Validator or node count
Each project tests with whatever it happened to have.
Geography
One data centre has no inter-continental latency and no packet loss.
Test duration
A short burst carries none of the storage, indexing or syncing growth.
Definition of a transaction
Nobody agrees, and it is rarely stated.
Reproducibility
Without open scripts and configs, nobody else can run it.
Change any one of these and the headline figure moves. Most published benchmarks vary several at once, which is why two numbers reported as TPS are not the same quantity.
Different Architectures Carry Different Security Guarantees, and That Changes What the Number Means
These three architectures don't just process transactions differently, they protect them differently with a different security protocol. A throughput number that ignores that difference is comparing systems that aren't actually offering the same thing.
A sharded network's transactions are typically still secured by validators drawn from and accountable to the same overall network, even though any single shard only sees a fraction of the total transaction load. A sidechain, by contrast, usually runs its own independent validator set and its own consensus mechanism, meaning its security is not directly inherited from the main chain it bridges to. This is a distinction repeatedly flagged in technical overviews of Layer 2 scaling: sidechains carry independent security models that can be meaningfully weaker than the chain they connect to. This is a tradeoff that needs to be evaluated with additional literature in the field. A payment channel's security rests on a different foundation still, cryptographic commitments and the ability to dispute an invalid state on-chain if a counterparty misbehaves, which works well for funds actively sitting in an open channel. What this doesn't address though, is that the liquidity of the money is locked up until delivery.
If we put a sidechain's throughput number next to a payment channel's throughput number next to a sharded network's throughput number, without accounting for the fact that one of them is trusting an independent validator set, one of them is trusting cryptographic dispute resolution between two specific parties, and one of them is trusting the same validator pool as the base layer, the comparison is simply not fair. There are way too many other factors at play for TPS to be the end-all, be-all determining factor that we treat it as.
Where does the security actually come from?
Sharding
The base network validator pool
Accountable to the whole network, though any one shard sees a fraction of the load.
Sidechain
Its own independent validator set
Not inherited from the chain it bridges to, and can be meaningfully weaker.
Payment channel
Cryptographic dispute on-chain
Between two specific parties, for funds sitting in an open channel.
Three different guarantees. A throughput number that ignores which one is in force is comparing systems that are not offering the same thing.
"Cost" and "Finality" Don't Mean the Same Thing Across Architectures
The same problem shows up in the other two numbers usually placed next to throughput: cost and finality.
A sidechain's per-transaction cost is usually just its own gas fee, but that ignores the separate cost of bridging value onto and off of the sidechain in the first place. This is a cost that doesn't exist for a sharded network processing a transaction natively within one chain. A payment channel's advertised near-zero fee ignores the cost of the capital sitting locked inside the channel the whole time. This produces a major opportunity cost as this money isn't being used to produce more money or for consumption. It is completely illiquid.
Finality is arguably worse. "Final" on a sharded network usually means the transaction has been committed within its shard and cross-shard confirmation has completed. "Final" on a sidechain can mean confirmed on the sidechain itself, which is a different and often weaker guarantee than confirmed and settled back on the main chain through its bridge. "Final" on a payment channel can mean the payment has been acknowledged instantly between the two parties in the channel, while the actual on-chain of that channel's balance doesn't happen until the channel closes, potentially much later. This once again, showcases the lack of standardization. A fair metric needs to be developed for finality as well for all 3 of these technologies to be compared against each other.
What has actually happened when a payment is called final?
Sharding
Committed in-shard, cross-shard confirmed
Cost is native to the chain, with no bridging step.
Sidechain
Confirmed on the sidechain itself
Weaker than settled back on the main chain. Bridging on and off is a separate cost.
Payment channel
Acknowledged between the two parties
On-chain settlement waits for the channel to close, and capital is locked until it does.
Cost has the same problem. A sidechain fee omits bridging, and a channel's near-zero fee omits the opportunity cost of capital sitting locked and illiquid.
What a Fair Comparison Actually Requires
All the points I made earlier in the blog post can lead you to think one thing and one thing only, these three technologies simply can't be compared. This is not true, however. Everything above illustrates that you can't compare these three technologies with published benchmarks due to a lack of standardization. What is possible, however, is publishing and testing your own benchmarks under a standardized methodology and procedure. This is what our lab is currently working on with our research paper.
At minimum, a comparison worth trusting needs matched hardware specifications across every architecture being tested, a comparable number of validators or nodes rather than whatever each project happened to test with, geographically realistic network conditions instead of a single data center, sustained load over hours or days rather than a short burst, a shared and explicit definition of what counts as one transaction, separate accounting for bridge, liquidity and a stated definition of finality for each architecture. Every one of these variables gets held differently, or not held at all, across most of the benchmark numbers currently circulating for these three approaches, which is exactly why the headline figures don't actually settle which architecture performs best for a remittance workload.
- 01Matched hardware specifications across every architecture tested
- 02A comparable number of validators or nodes, not whatever each project used
- 03Geographically realistic network conditions, not a single data centre
- 04Sustained load over hours or days, not a short burst
- 05A shared, explicit definition of what counts as one transaction
- 06Separate accounting for bridge and liquidity costs
- 07A stated definition of finality for each architecture
Every one of these is held differently, or not at all, across the benchmarks currently circulating. That is the gap the lab's own paper is built to close.
Sources
- 1.ACM, "BBSF: Blockchain Benchmarking Standardized Framework," on the lack of standardized metrics and workloads across blockchain performance evaluations: dl.acm.org/doi/fullHtml/10.1145/3595647.3595649
- 2.a16z crypto, "Why blockchain performance is hard to measure," on the absence of standardized metrics and the problem of treating all transactions as equal: a16zcrypto.com/posts/article/why-blockchain-performance-is-hard-to-measure
- 3.Celestia, "Why Blockchain Benchmarks Are Usually Deceiving," on short-burst benchmarks versus sustained production load: blog.celestia.org/why-blockchain-benchmarks-are-usually-deceiving
- 4.BNB Chain Blog, "Designing Benchmarks for Trading-Focused Blockchains," on objective, verifiable, and transparent benchmarking design principles: bnbchain.org/en/blog/designing-benchmarks-for-trading-focused-blockchains
- 5.Easy Crypto / Zilliqa project history, on Zilliqa's 2017 testnet results of 2,488 TPS using 3,600 nodes on AWS Singapore: hub.easycrypto.com/zilliqa-coin and blog.zilliqa.com, "Zilliqa Testnet v1.0 Release: Codename Red Prawn"
- 6.iCryptoAI, "Scalability Solutions for Blockchain: Sharding and Layer-2 Technologies," on sidechains carrying independent, potentially weaker security models than the main chain: icryptoai.com/2025/11/26/scalability-solutions-for-blockchain-sharding-and-layer-2-technologies
- 7.European Commission Blockchain Observatory, "An overview of blockchain scalability, interoperability and sustainability," on sharding, off-chain payment channels, and cross-shard communication challenges: blockchain-observatory.ec.europa.eu
RemitBench Methodology · System Comparison
