Skip to main content

Market Mechanics

This reference covers the mechanical foundations of electronic markets: how exchanges process orders, how matching algorithms determine execution priority, how market data is generated, and why these details matter for Bookmap add-on development. Every concept is structured as a discrete unit with explicit cause→effect relationships.

1. The Core Axiom​

Price is 100% determined by the orders of traders — and nothing else.

This is not a simplification. It is the literal, mechanical truth of how markets work:

  1. A trader sends an action (new order, cancel, modify) to the exchange.
  2. The exchange's matching engine processes the action using a public, deterministic algorithm.
  3. The matching engine updates the order book.
  4. The exchange generates market data describing what changed.
  5. All indicators — price, volume, VWAP, candlesticks, everything — are derived from this market data.

Key implication: When reasoning about "why price moved," the only valid causal chain is: trader action → matching engine processing → order book change → market data update. External events (wars, earnings, Fed decisions) only matter because they change traders' decisions, which change their orders. The event itself does not touch the matching engine.

2. The Exchange Architecture​

┌─────────────────────────────────────────────┐
│                  EXCHANGE                    │
│                                             │
│   ┌──────────────────┐                      │
│   │ Sync & Admin     │ ← Atomic ordering    │
│   └────────┬─────────┘   of all actions     │
│            ▼                                │
│   ┌──────────────────┐                      │
│   │ Matching Engine   │ ← Deterministic     │
│   │  ┌─────────────┐ │   algorithm          │
│   │  │ Order Book   │ │                      │
│   │  └─────────────┘ │                      │
│   │  ┌─────────────┐ │                      │
│   │  │Data Generator│ │ → Market Data out   │
│   │  └─────────────┘ │                      │
│   └──────────────────┘                      │
└─────────────────────────────────────────────┘
         ▲                        │
    Actions IN              Market Data OUT
  (send/cancel/modify)    (executions + book updates)
         │                        │
         │                        ▼
  ┌──────────┐            ┌──────────────┐
  │ Traders  │            │ Data Vendors │
  │ (via     │            │ (redistribute│
  │ brokers) │            │  to clients) │
  └──────────┘            └──────────────┘

Key point: Every trader maintains a local copy of the order book, reconstructed from the market data stream. This copy may lag the exchange's true book by the trader's latency.

3. The Three Trader Actions​

There are only two or three fundamental actions a trader can perform. Everything else is a variation.

#ActionParametersExchange Response
1Send a new orderBuy/Sell, Quantity, Limit PriceOrder ID (assigned by exchange)
2Cancel an orderOrder IDConfirmation or rejection
3Modify an orderOrder ID, New Quantity, New Limit PriceConfirmation or rejection

Why Modify Exists (Not Just Cancel + Re-Send)​

Modify is technically redundant (cancel + new order achieves the same result), but it solves two problems:

  • Latency reduction: Both sub-actions execute atomically inside the exchange — no round-trip gap.
  • In-flight risk elimination: If you cancel then re-send as two separate messages, the original order might fill between your cancel arriving and your new order arriving, leaving you with an unintended execution AND a new order. Modify avoids this.
  • Queue preservation: If the modification only reduces size at the same price, the order keeps its position in the queue. Any price change or size increase sends it to the back of the queue.

Key insight: When analyzing order book changes via MBO data and you see a replace event at the same price with smaller size → the order kept its queue priority. If you see a replace with a different price → it lost priority (equivalent to cancel + new send).

4. Order Types​

4.1 Market Order​

  • Parameters: Buy/Sell + Size
  • Behavior: Executes immediately against resting orders on the opposite side, sweeping through price levels until filled.
  • No price protection — you get whatever the book offers.

4.2 Limit Order​

  • Parameters: Buy/Sell + Size + Price
  • Behavior: "Do not execute at a worse price than specified."
  • If the limit price allows immediate matching → partial or full fill, remainder rests in book.
  • If no immediate match → order rests in book at the specified price.

4.3 Iceberg Order​

  • Adds: Maximum displayed size (the "tip")
  • Behavior: Only shows a fraction of the total size in the visible order book. As the displayed portion fills, the hidden portion refills the displayed amount.
  • Detection implication: Repeated refills at the same price level at consistent size intervals signal iceberg activity. With MBO data, you can track the order ID and observe refills directly.

4.4 Stop Order​

  • Adds: Stop price (trigger condition)
  • Behavior: The order is invisible in the order book. When a trade occurs at or worse than the stop price, the exchange publishes/releases the order (typically as a market order or limit order).
  • Key insight: Stop orders are part of the invisible conditional order pool. They do not appear in MBP or MBO data until triggered. This is why "stop hunts" work — the triggering trader cannot see where stops are, but can infer their location from price behavior patterns.

4.5 Other Conditional Orders​

Exchanges offer many variations: time-in-force conditions (GTC, IOC, FOK, DAY), order-sends-order chains, and custom types. Brokers and OMS platforms add even more flexibility on top, managing orders from co-located facilities with sub-millisecond response times.

5. The Order Book​

The order book has two parts:

5.1 Visible Part (What Market Data Shows)​

  • Collection of resting limit buy orders (the bid book) and limit sell orders (the ask book).
  • These orders rest because their limit price has not yet permitted a match.
  • Orders at the same price form an order queue, with priority determined by the matching algorithm.

5.2 Invisible Part (Conditional Orders)​

  • Stop orders, hidden iceberg portions, time-conditional orders.
  • These exist at the exchange but are not reflected in any market data until their trigger condition is met.
  • Key insight: The visible book is always an incomplete picture. Significant liquidity may be hiding as stops, icebergs, or other conditional orders. Never assume the visible book represents total supply/demand.

5.3 Order Queue Mechanics​

  • Orders advance in the queue when orders in front of them are cancelled or executed.
  • Queue position is valuable — it determines execution priority.
  • The matching algorithm determines how the queue is structured and how priority is assigned.

6. Market Data Types​

6.1 Market by Price (MBP)​

  • Shows the total aggregate size at each price level.
  • May be full depth or limited (top 5, 10, or 20 levels).
  • You see: "There are 350 contracts bid at 5950.00" — but NOT how many individual orders make up those 350.
  • Limitation: Cannot distinguish between one order of 350 and thirty-five orders of 10. Cannot detect individual order behavior (spoofing, icebergs, queue dynamics).

6.2 Market by Order (MBO)​

  • Shows individual orders with unique Order IDs, their price, size, and queue position.
  • Provides full market depth.
  • You see: "Order #A7F3 is a bid for 25 contracts at 5950.00, position 3 in queue."
  • This is the more modern, more transparent data type.
  • Enables: Iceberg detection, spoofing analysis, queue position tracking, individual order lifecycle observation.

6.3 The Data Hierarchy (What You Lose at Each Level)​

MBO Data (100% of available information)
  │
  │ Aggregate by price level, discard order IDs
  ▼
MBP Data (~30-50% of information)
  │
  │ Discard book, keep only trades
  ▼
Times & Sales / Last Trade (~5% of information)
  │
  │ Aggregate by time period
  ▼
Candlestick Charts (~1-2% of information)

Key insight: A candlestick is a lossy compression of the actual market activity. Any analysis done on candlesticks alone is working with roughly 1-2% of the available signal. Order flow analysis (MBO/MBP) recovers 50-100% of the signal.

7. Market Maker vs. Market Taker​

Every trade has exactly two sides:

RoleDefinitionTypical CommissionBehavior
Market MakerThe resting (passive) limit order that was already in the bookLower or even negative (exchange pays you)Provides liquidity, earns spread + rebate
Market TakerThe newly arrived (aggressive) order that matches against the resting orderHigherConsumes liquidity, pays spread + commission

Key insight: The commission structure directly shapes market behavior. When maker rebates are high (like BitMEX's -0.025% maker / 0.075% taker), traders compete intensely to be the maker rather than taker. This causes:

  • Thick resting liquidity at popular levels (everyone wants to be the passive side)
  • Price tends to move in discrete jumps rather than smooth ticks (takers need enough expected move to overcome the cost)
  • Aggressive "queue competition" where traders race to place limit orders early

Key insight: When analyzing any market, always consider the commission structure. It determines whether the dominant behavior will be aggressive (taker-friendly markets) or passive (maker-friendly markets).

8. Matching Algorithms​

8.1 FIFO (Price-Time Priority)​

The most widely used algorithm. Used by CME for E-mini S&P 500 (ES) futures.

Rules for processing a NEW order:

  1. Match/execute against resting orders on the opposite side:
    • As long as the limit price permits (no limit for market orders)
    • As long as the incoming order has remaining size
  2. Add any unfilled remainder to the order book at the end of the queue at the order's limit price.

Rules for processing a CANCEL:

  1. Cancel the remaining unfilled part, or reject if the order no longer exists.

Rules for processing a MODIFY:

  1. Reject if the order no longer exists.
  2. If the only change is reducing size at the same price → adjust size, keep queue position.
  3. Otherwise → remove from queue, process as a new order with modified price/size (loses queue position).

Implication: In FIFO markets, queue position is extremely valuable. Being first in the queue at a price level means you get filled before everyone else at that price. This creates:

  • Incentive to place orders early at anticipated support/resistance levels
  • "Queue scalping" strategies where the edge is purely in queue position
  • Aggressive cancellation when the queue position is no longer favorable

8.2 Pro-Rata​

Matches orders proportionally based on their size rather than arrival time.

  • A large order at the back of the queue can still get a proportional fill.
  • Encourages posting larger size (bigger share of the pro-rata allocation).
  • Used in some interest rate and options markets.

8.3 FIFO with LMM (Lead Market Maker)​

Enhanced FIFO that gives registered Lead Market Makers priority allocations before the standard FIFO queue is processed. This means:

  • LMMs get filled first at a price level, regardless of when they placed their order.
  • Regular participants only get fills from the remainder.
  • This is an explicit structural advantage for designated liquidity providers.

Key insight: Always know which matching algorithm applies to the instrument being traded. The algorithm fundamentally determines what constitutes an "edge" — in FIFO markets, speed and queue position matter most; in Pro-Rata markets, size matters most.

9. Latency​

9.1 Three Components of Latency​

Total Trading Cycle Latency:

1. Market Data Latency
   Exchange → Your System
   (receiving the information that something changed)

2. Processing Latency  
   Your System internally
   (deciding what to do about it)

3. Round-Trip Time (RTT)
   Your System → Exchange → Your System
   (sending your action and receiving confirmation)

9.2 Why Latency Matters​

  • Faster traders capture short-term opportunities before slower traders.
  • If two traders identify the same opportunity, the faster one gets a better execution price.
  • The slower trader either gets a worse price or misses the opportunity entirely.

9.3 Latency Distribution Is NOT Uniform​

This is a critical insight for simulation/backtesting:

  • Average latency might be ~1 millisecond for a co-located system.
  • 99.99th percentile might be 100+ milliseconds.
  • These extreme outliers are NOT random — they are highly correlated with bursts of market activity (when the matching engine is overloaded).
  • This means: latency spikes happen exactly when you most need low latency (during volatile events when many strategies spot the same opportunity simultaneously).

Key insight: Never assume constant latency in simulations. Never assume latency outliers are randomly distributed. The correlation between latency spikes and market volatility is a fundamental structural feature of electronic markets.

9.4 Artificial Speed Bumps​

Some exchanges deliberately add latency delays (e.g., 350 microseconds) for regular participants while allowing designated market makers to bypass the delay. This:

  • Gives structural advantage to preferred market participants
  • Is publicly documented in exchange rules
  • Is conceptually similar to giving LMMs queue priority — just implemented via timing rather than matching logic

10. The Two Fatal Simulation Mistakes​

10.1 Mistake: Wrong Latency Assumptions​

Problem: Assuming constant average latency (e.g., 1ms) in backtests.

Why it's fatal: The most profitable-looking signals in historical data often coincide with the moments of highest latency (market activity bursts). A backtest using average latency will show great fills at these moments. In reality, your order would arrive 100ms+ late, getting a much worse fill or no fill at all.

ML/AI trap: Pattern detection algorithms will preferentially identify "obvious" opportunities that many participants simultaneously detect — precisely the moments when latency spikes and execution quality collapses.

10.2 Mistake: Ignoring Own Market Impact​

Problem: Assuming your simulated trades would not have changed the market data you're backtesting against.

Why it's fatal: Every order affects the market. Even small orders trigger reactions from other participants (market makers adjusting, other algos responding). The cumulative effect of your activity would have produced different market data than what you observed historically.

Chaotic system analogy: Markets are chaotic systems where infinitesimal differences in starting conditions lead to drastically different outcomes over time. Your backtest is simulating trades in a universe that would not have existed if those trades had actually occurred.

Key insight: The only valid test of a strategy is live trading or an interactive simulation where other participants react to your actions. Static historical backtesting always overstates performance.

11. Information Hierarchy for Trading Decisions​

DATA (raw facts)
  │ Contains: useful signals, irrelevant noise, illusions, deceptions
  │
  │ Structure and filter
  ▼
INFORMATION (organized, meaningful data)
  │
  │ Interpret in context
  ▼
KNOWLEDGE (contextual understanding of patterns and relationships)
  │
  │ Test and validate through experience
  ▼
WISDOM (validated knowledge that guides effective action)
  │
  │ Form intentions
  ▼
DECISIONS (specific plans based on wisdom)
  │
  │ Execute
  ▼
ACTIONS (orders sent to exchange)
  │
  │ Observe outcomes
  ▼
FEEDBACK → loops back to DATA

Key insight: Market data is at the DATA level. It contains noise, illusions (spoofing), and deceptions alongside genuine signals. The agent's job is to help the trader ascend this hierarchy — not just relay raw data, but help convert it into knowledge and wisdom.

12. Why "Wars Don't Affect the Market" (Directly)​

This is a philosophical but practically important point:

  • No external event directly touches the matching engine.
  • Events affect traders' decisions, which affect their orders, which affect the order book, which affects price.
  • The causal chain always passes through human (or algorithmic) decision-making.
  • This is why watching order flow (the actions of traders) is, in Bookmap's words, "the fastest global news feed" — you see the reaction to events in real-time, before the news even reaches most people, by observing how market makers and HFT firms reposition.

Practical implication: Rather than trying to predict what events will happen, observe how participants with the fastest information are actually positioning. Their orders are their "vote" backed by real money — more reliable than any analyst's opinion.

13. Key Principles Summary​

  1. All price movement originates from orders. There is no other mechanism.
  2. The matching algorithm is public and deterministic. Given the same sequence of actions, the same result always occurs.
  3. The visible order book is incomplete. Conditional orders (stops, icebergs, time-based) are invisible until triggered.
  4. MBO > MBP > Times & Sales > Candlesticks in information content. Always use the highest-resolution data available.
  5. Commission structure shapes behavior. Maker-friendly markets behave differently from taker-friendly markets.
  6. Queue position matters in FIFO markets. Being early at a price level is itself an edge.
  7. Latency spikes are correlated with opportunity. The best signals are hardest to execute.
  8. Backtests always lie. They ignore latency reality and own market impact.
  9. Order flow is the fastest news feed. Watching what traders do with money beats watching what analysts say.
  10. Markets are interactive, chaotic systems. Every action changes the system for all other participants.

14. Glossary​

TermDefinition
Aggressive orderA newly arrived order that immediately matches against resting orders. Synonymous with "market taker."
Ask bookThe collection of resting sell limit orders, sorted by price ascending.
Bid bookThe collection of resting buy limit orders, sorted by price descending.
Best bid / Best askThe highest bid price and lowest ask price currently in the book. The "inside market."
SpreadBest ask minus best bid. The cost of immediately crossing from one side to the other.
DepthThe number of price levels (or total size) available in the order book beyond the best bid/ask.
FIFOFirst-In-First-Out. Price-time priority matching algorithm. Orders at a price level are filled in the order they arrived.
IcebergAn order where only a fraction of the total size is visible. The hidden portion refills the displayed amount after fills.
LMMLead Market Maker. A designated participant with structural advantages (queue priority, speed bump exemptions).
Market dataThe output stream from the exchange describing all changes to the order book and all executions.
Matching engineThe exchange component that processes trader actions against the order book using the matching algorithm.
MBOMarket by Order. Data feed showing individual orders with unique IDs. Highest information content.
MBPMarket by Price. Data feed showing aggregate size at each price level. Less information than MBO.
Order bookThe collection of all resting limit orders at the exchange, organized by side (bid/ask) and price.
Order queueAll orders at a single price level, arranged by the matching algorithm's priority rules.
Passive orderA resting limit order in the book. Synonymous with "market maker."
Pro-RataMatching algorithm that allocates fills proportionally based on order size rather than arrival time.
Queue positionAn order's place in the priority sequence at its price level. Determines fill order in FIFO markets.
RTTRound-Trip Time. Time from sending an action to receiving the exchange's response.
Speed bumpAn artificial delay imposed by some exchanges on certain participants' actions.
Stop huntPrice movement specifically aimed at triggering clusters of invisible stop orders, which then cascade into further movement.
Times & SalesTrade-only data feed. Shows executed trades without order book context. ~5% of available information.

See Also​