Dear reader,
This blog goes a bit deeper than the previous ones.
If you enjoy understanding how systems behave under pressure, how latency really works, and what happens inside a trading engine during volatility, you will feel at home here. If not, that is completely fine too.
We are going to explore latency distributions, determinism, and why many trading systems struggle precisely when markets become unpredictable. More importantly, we will explain how Hyperprop is engineered to solve that problem at its core.
Welcome in the engine room.
Latency Is a Distribution, Not a Number
Most platforms advertise a single metric.
“Execution in 1 ms.”
That number is usually p50, the median. It reflects calm conditions.
Real systems produce a latency distribution. Most requests cluster at low values, and a small percentage stretch out to the right. That right side is the tail.
When we say p99.9, we mean the latency below which 99.9 percent of requests complete. The remaining 0.1 percent are slower.
In trading, those outliers matter more than the median. They tend to appear during burst traffic and market volatility, exactly when capital is moving.
Engineering for p50 makes dashboards look good.
Engineering for p99.9 keeps systems stable under pressure.
Where Latency Comes From
An order is not a single operation. It passes through layers:
- Network ingress
- Authentication
- Risk evaluation
- Matching or routing
- State mutation
- Persistence
- Client notification
Each step consumes part of your latency budget.
Under load, additional effects appear:
- Lock contention
- Allocation churn
- CPU scheduling delays
- Blocking I/O
- Network jitter
These do not scale linearly. Small inefficiencies compound. The tail grows faster than the median moves.
Designing for Determinism
In a trading engine, being predictable is more important than being fast on average.
The real danger is not that the system is slightly slower one day. The real danger is that it suddenly pauses when market activity explodes.
That is why we chose Rust.
Many popular backend languages such as Java, C#, and Go use something called a garbage collector. This automatically cleans up memory while the program is running. Most of the time this works well. But occasionally, the runtime pauses briefly to reorganize memory.
Those pauses are usually small. Under heavy load, they become larger and less predictable. They show up in the tail of the latency distribution.
In trading systems, that tail is what matters most.
Rust does not use a garbage collector. Memory management is explicit and controlled at compile time. That gives us:
- No surprise pauses caused by memory cleanup
- Clear and predictable allocation behavior
- No runtime deciding to interrupt execution
- Concurrency guarantees enforced before the code ever runs
This keeps the execution path simple.
When an order enters the engine, the critical path does only three things:
- Validate the order
- Update in memory state
- Emit an event
Nothing more.
Everything else is separated from that path.
Database writes happen asynchronously.
WebSocket updates are handled through an internal event system.
If a client connection is slow, it cannot block order execution.
The engine owns the live state in memory. Other systems observe it, but they do not control its timing.
Rust was not chosen because it is trendy.
It was chosen because it removes an entire category of unpredictable runtime behavior.
Physics Still Wins
Even perfect software cannot overcome physical distance.
Data traveling between continents takes measurable time. That cannot be optimized away.
Infrastructure must be deployed close to users. Once the engine itself is efficient, network stability becomes the dominant factor in how fast the platform feels.
In many cases, inconsistent latency is more harmful than slightly higher but stable latency.
Observing the Tail
So if you want to control p99.9 latency, you need visibility.
We measure:
- How long each stage of order processing takes
- Queue depth and pressure in the system
- Network round trip times
If latency increases by 40 milliseconds, we can identify exactly where that time was spent.
Without this level of insight, tail latency becomes guesswork.
Why It Matters
In a consumer app, a slow request is frustrating.
In proprietary trading, a slow request during volatility can lead to slippage, delayed rule enforcement, or loss of confidence in the platform.
Trust is built on stability under stress.
Hyperprop is designed around that principle.
Not optimized for averages.
Optimized for the moments that matter most.