Introduction¶
Chapter Optimal Execution Theory addressed the scheduling problem: given a large order to execute over a horizon , what sequence of child order sizes minimises expected cost for a given level of risk aversion? The Almgren–Chriss model and its extensions produce optimal schedules that specify, at each time step , a target volume to trade in the next slice .
A slice, however, is not an execution. Sending a market order of size to the exchange is always an option, but it is rarely optimal: it consumes the spread and accelerates local price impact within the slice. Execution tactics are the micro-level algorithms that implement each slice in the limit order book, deciding dynamically whether to post a limit order (and at what price level), upgrade to a market order, or split the slice across multiple venues. Where strategies resolve the tension between market impact and timing risk at the scale of minutes to hours, tactics resolve the analogous tension between execution probability and price improvement at the scale of seconds to minutes.
This chapter develops the theory and practice of execution tactics. Section Fill probability model recalls the fill probability model from Chapter Modelling the Limit Order Book — the core stochastic ingredient for all tactic design. Section Single-market execution tactic formulates the single-market placement problem as an optimal stochastic control problem, derives the Hamilton–Jacobi–Bellman equation, and characterizes the solution as an aggressiveness matrix indexed by remaining time and remaining quantity. Section Smart order routing extends the framework to smart order routing across multiple venues. Section Heuristic execution tactics describes the menu of heuristic tactics offered by brokers and exchanges, and section Reinforcement learning for execution tactics develops the reinforcement learning approach to tactic design, which bypasses the need for explicit model calibration.
Fill probability model¶
Section Probability of filling a limit order of Chapter Modelling the Limit Order Book develops the fill probability model in full, including derivation, calibration, extensions with additional LOB features, and monotonicity constraints. Here we recall the key result needed for tactic design.
Following Avellaneda and Stoikov Avellaneda & Stoikov, 2008 and Cont, Stoikov, and Talreja Cont et al., 2010, fill events for a limit order placed ticks from the mid-price are modelled as a Poisson process with depth-dependent intensity:
where is the base fill rate and is the depth sensitivity. The probability of a fill within a time window is:
In practice, and are calibrated from historical LOB data by regressing observed fill rates against placement depth, controlling for spread, order imbalance, and time of day (see Section Probability of filling a limit order). Typical values for liquid equity markets are –2.0 fills per minute at zero depth and –3 per tick.
Single-market execution tactic¶
Problem setup¶
We now formulate the tactic’s control problem. The tactic is responsible for executing shares (a sell order, without loss of generality) within a window . The state at time is the remaining quantity ; the control is the placement depth at which to post a limit sell order.
Dynamics. Fills arrive as a Poisson process with rate ; each fill reduces the remaining quantity by one unit:
We normalize the tick size to one, so a fill at depth yields proceeds of ticks above the mid-price.
Terminal condition. If any quantity remains at , it must be sold immediately via a market order. The forced liquidation incurs a cost of ticks per unit below mid-price:
The parameter is a market property — the effective half-spread plus market impact of an unplanned emergency order — estimated from historical trade data during the calibration period alongside and . It is not a free parameter that the tactic designer controls.
Objective. The tactic maximises the expected CARA (Constant Absolute Risk Aversion) utility of total proceeds, with risk aversion coefficient :
where is total proceeds including the terminal liquidation. The risk aversion parameter is the tactic’s aggressiveness control: a high- agent dislikes the possibility of reaching the deadline with unsold inventory and posts aggressively to ensure execution; a low- agent (near risk-neutral) prioritises price improvement and posts passively. This formulation is consistent with the exponential utility framework used in the market-making chapters.
Certainty equivalent. Working directly with the exponential utility is cumbersome. We transform to the certainty equivalent value function, which expresses values in the same units as raw proceeds (ticks):
The boundary conditions become (terminal liquidation cost) and (no inventory, no further proceeds).
The HJB equation and optimal placement¶
Let denote the minimised disutility (, related to by ). Applying Bellman’s principle over and taking , the Hamilton–Jacobi–Bellman equation for is:
Setting the derivative with respect to to zero gives the first-order condition:
Converting to the certainty-equivalent (using ) and defining:
the optimal placement depth is:
The first term, , is the risk-comfort depth: the base placement a risk-averse agent maintains even without inventory urgency. It replaces the of the risk-neutral case, and has three key properties:
Recovery: as , — the risk-neutral result is recovered.
Monotonicity: the risk-comfort depth is strictly decreasing in — greater risk aversion always tightens the placement relative to the risk-neutral benchmark.
Limit: as , the risk-comfort depth — an extremely risk-averse agent posts at the best quote regardless of price improvement.
The interpretation of is unchanged: the optimal placement depth equals the risk-comfort depth minus the absolute urgency . A market order is optimal when , i.e., when urgency exceeds the risk-comfort depth.
Substituting the optimal back and transforming to gives the reduced ODE in remaining time :
The constant encodes both the market fill-rate parameters and the agent’s risk aversion . As , , recovering the risk-neutral ODE. The structure of the equation — and hence the backward induction algorithm — is identical to the risk-neutral case; only the driving constant changes.
Analytical solution for . When only one unit remains, , so and the ODE becomes:
This separable ODE has the solution (with initial condition ):
The optimal placement depth for the last unit is:
As : , which triggers a market order when — a condition that is increasingly easy to satisfy as increases. As : grows without bound — with ample time, even a risk-averse agent posts very passively. For , the system is solved numerically by backward induction on the grid .
The aggressiveness matrix¶
The solution is most naturally displayed as a two-dimensional look-up table indexed by the normalised remaining time and the normalised remaining quantity — the aggressiveness matrix of the tactic. Each cell of the matrix specifies the optimal placement depth in ticks at that (time, quantity) state.

Figure 1:Aggressiveness matrices for three levels of risk aversion (low: , moderate: , high: ; with fill/min, per tick, tick calibrated, min). Colour indicates optimal placement depth in ticks: darker (blue) regions correspond to deeper, more passive placement; lighter (yellow/red) regions trigger aggressive or market orders. In all cases, the tactic becomes more aggressive as time expires or as the remaining fraction approaches 1. A higher risk aversion shifts the entire matrix towards more aggressive placements, shrinking the risk-comfort depth and enlarging the market-order region.
Several properties of the aggressiveness matrix are economically intuitive:
Top-left corner (early, little remaining): the tactic posts deep passive orders, harvesting price improvement with minimal urgency.
Bottom-right corner (late, most remaining): the tactic posts at or near the best quote, maximising fill rate at the cost of price improvement.
Market order region: the shaded region where corresponds to states where the risk of terminal liquidation is so imminent that immediate execution via a market order is optimal. This region grows with : a more risk-averse agent accepts market orders in more states.
In practice, the matrix is pre-computed off-line using backward induction and stored as a look-up table. At each step of the tactic, the current state is mapped to the nearest grid cell, and the corresponding depth is read off. This makes real-time execution computationally trivial.
Multi-order splitting¶
The single-order formulation posts one limit order at a time. In practice, it is common to split the remaining quantity into simultaneous child orders, each of size , posted at (potentially different) price levels. The HJB equation for the multi-order case with simultaneous orders is:
Because the objective is separable — each order’s contribution depends only on its own and the common term — the optimal are identical for all :
All orders are placed at the same optimal depth as in the single-order case Cartea & Jaimungal, 2015. The only effect of splitting is to multiply the effective fill rate: with orders at depth , the aggregate fill rate is , reducing the expected time to the next fill by a factor of . The aggressiveness matrix is therefore unchanged; only the inventory dynamics change (fills arrive times faster on average). Splitting reduces the risk of the tactic finishing the window with large residual inventory, at no cost to price improvement.
Smart order routing¶
Multi-venue framework¶
Modern financial markets are fragmented: the same instrument trades simultaneously on multiple lit exchanges, dark pools, and bilateral platforms with different liquidity profiles. A Smart Order Router (SOR) is a component that distributes the tactic’s child orders across these venues, seeking to fill the required volume at the lowest total cost.
We model trading venues, each characterized by its own depth-dependent fill rate:
where is the base fill rate at venue and controls how steeply fill rates fall with depth. Venues with high (high volume) and low (book does not thin rapidly) are most liquid. The tactic simultaneously posts one limit order per venue, with depth at venue .
Optimal venue allocation¶
With multiple venues, the HJB equation becomes:
The multi-venue objective is again separable across venues for a given . The first-order condition for each venue independently yields:
The optimal depth at each venue follows the same formula as the single-venue case, with the risk-comfort depth being venue-specific through . This has a direct economic interpretation: venues where fill rates fall off slowly with depth (small ) attract deeper, more passive placements, while venues where fill rates fall off rapidly (large ) attract more aggressive placements. Additionally, for a given , depth is tighter everywhere compared with the risk-neutral benchmark.
The substituted HJB becomes:
The SOR aggregates CARA-adjusted fill rate contributions from all venues simultaneously, without requiring an explicit allocation of volume to each venue.

Figure 2:Smart order routing across two venues. Left: fill rates for a liquid venue (high , low ; blue) and an illiquid venue (low , high ; orange). The liquid venue attracts deeper passive placements; the illiquid venue is approached more aggressively. Right: simulated P&L distributions (10,000 runs, , min) comparing single-venue execution (liquid, illiquid) against the two-venue SOR. The SOR dominates both single-venue alternatives in both mean and variance.
Practical considerations¶
Beyond the theoretical model, SOR design involves several practical dimensions:
Dark pools. Dark pools are off-exchange trading venues that do not display their order book; fills occur at the mid-price without market impact, but fill rates are uncertain and typically lower than on lit venues. The fill rate model for dark pools requires a different calibration — fills are generated by crossing contra-side orders, not by price movement. The seminal analysis of the dark pool problem is due to Ganchev et al. (2010), who frame it as a censored exploration problem.
Latency and queue position. Fill rates in limit order books depend not only on depth but on queue position at each level. An order that arrives early at a given price level will be filled before later arrivals at the same level (price-time priority). The model above implicitly assumes the order is at the back of the queue; incorporating queue position leads to a richer state space and has been studied in the literature on optimal placement with latency Cont et al., 2010.
Price divergence across venues. In fragmented markets, mid-prices can temporarily diverge across venues, especially at high frequency. A SOR that ignores this may place its order at the “wrong” venue. Best-execution regulations (MiFID II in Europe, Regulation NMS in the US) require systematic evaluation of execution quality across venues and oblige brokers to seek the best available price.
Heuristic execution tactics¶
Beyond the model-based approaches above, a range of heuristic tactics are widely used in practice, supported natively by exchanges and prime brokers. These tactics encode practical execution intuitions that are simpler to implement than full HJB solutions, and serve as useful benchmarks.
Pegged orders¶
A pegged order tracks a reference price dynamically, automatically re-pricing the order as market conditions change. The most common variants are:
Peg-to-mid: the limit order is always posted ticks above (for a sell) or below (for a buy) the current mid-price. As the mid moves, the order reprices automatically, maintaining a constant spread to the centre of the market.
Peg-to-best (NBBO peg): the order is pegged to the national best bid/offer, posting at ticks above the best ask (for a sell) or below the best bid (for a buy). This keeps the order at a fixed distance from the best prevailing market price rather than the mid.
Primary peg: the order is always posted exactly at the current best bid (for a sell) or best ask (for a buy), ensuring it is the first passive quote available.
Pegged orders reduce the operational complexity of managing limit orders in fast-moving markets: without pegging, a passive limit order placed at the mid at time may be far inside the book (or even outside the book entirely) by time if the price has moved. Pegging ensures the order remains continuously visible at the intended level without manual re-pricing.
Iceberg and reserve orders¶
A large passive limit order that is fully displayed in the order book signals the trader’s intention to other market participants, who may adjust their own behaviour — pulling displayed orders, widening spreads, or trading against the disclosed position — to extract rents from the informed side. Iceberg orders (also called reserve orders or hidden quantity orders) mitigate this information leakage by displaying only a small “tip” in the public order book while reserving the remainder invisibly. When the tip is fully filled, the next tranche is automatically posted.
The trade-off in choosing the tip size is as follows. A larger tip signals more information to the market but earns better queue priority (since the displayed tip competes for fills on a first-come, first-served basis with its full size). A smaller tip minimises information leakage but may execute more slowly if queue priority is lost each time the tip is exhausted. Empirical evidence suggests iceberg orders execute at slightly better prices than equivalent market orders but slower than equivalent fully-disclosed limit orders, reflecting the information leakage channel Cartea et al., 2015.
Liquidity seeking and sniping¶
Liquidity seeking tactics scan multiple venues simultaneously for executable liquidity at or better than a target price. When liquidity is detected — a visible limit order on the contra-side at a favourable price — the SOR fires a market order (or a marketable limit order) to capture it before it disappears. These tactics are opportunistic: they are passive in the absence of favourable liquidity and aggressive when it appears.
Sniping is a closely related strategy in which the tactic waits for a specific price level to be reached and then submits an aggressive order. In the context of execution tactics, a sniper may target a pre-specified price (for a buy) below which it activates, treating the price as a favourable entry point. The risk of sniping is adverse selection: the price may be moving rapidly to the target level precisely because informed traders are selling, making the achieved entry point worse in expectation than the target.
Participation rate tactics¶
Participation rate (PoV) tactics constrain the execution rate to a fixed percentage of market volume, so that the tactic does not trade significantly faster than the prevailing market pace. Within each measurement interval , the tactic computes the market volume and submits enough orders to have traded shares in aggregate. If ahead of pace, it switches to passive limit orders; if behind pace, it switches to more aggressive orders or market orders.
PoV tactics are natural for VWAP strategies (see chapter Optimal Execution Theory): a constant participation rate (where is the expected daily volume) approximates the VWAP schedule. In practice, the participation rate is adjusted intraday based on updated volume predictions from the volume models discussed in chapter Modelling the Limit Order Book.
Reinforcement learning for execution tactics¶
RL formulation¶
The optimal placement problem of section Single-market execution tactic rests on explicit assumptions: Poisson fill processes with exponential depth dependence, a known terminal liquidation cost, and a model that is stationary within the execution window. In practice, fill rates vary with LOB state, intraday patterns, and recent order flow in ways that are difficult to capture parsimoniously. Reinforcement learning (RL) provides an alternative that learns the optimal policy from simulated or historical experience, without requiring explicit model specification.
The tactic is formulated as a Markov Decision Process (MDP):
State : remaining time , remaining fraction , recent volatility estimate , order flow imbalance , and estimated fill rate .
Action : the placement depth in ticks. Action corresponds to a market order; is at the best ask (for a sell); is ticks above the best ask.
Reward : proceeds from any fill at step , minus the terminal liquidation cost at (with the calibrated market parameter). Formally:
This linear reward corresponds to the expected-proceeds (risk-neutral) objective — the limit of the CARA criterion. The RL agent therefore learns to approximate the risk-neutral limit of the HJB policy. The theoretically grounded approach to risk-averse execution is to use the CARA HJB directly with an explicit , but the RL framework is valuable precisely when the state space is richer than and the fill rate model is non-stationary or complex.
The RL agent learns a policy that maximises the expected discounted sum of rewards (where is the discount factor) by estimating from experience rather than from explicit HJB solution.
The RL framework is particularly powerful when (i) the state space is richer than just — including LOB features that are difficult to model analytically — and (ii) the fill rate model is non-stationary or has complex dependence structure.
Q-learning and deep Q-networks¶
The seminal application of RL to execution tactics is due to Nevmyvaka, Feng, and Kearns Nevmyvaka et al., 2006, who applied tabular Q-learning to the order placement problem on NASDAQ data. Their state representation included remaining quantity, remaining time, bid-ask spread, and short-term price momentum. The action space was a discretized set of placement levels. Evaluated on 1.5 years of NASDAQ data, the RL policy outperformed TWAP by approximately 50 basis points on average for large-cap stocks — a significant improvement for institutional-scale executions.
In the tabular Q-learning approach, the action-value function is maintained as a look-up table and updated via the Bellman recursion:
where is the learning rate and is the discount factor. The policy is -greedy: with probability a random action is taken (exploration), and with probability the greedy action is selected.
When the state space is continuous or high-dimensional, the tabular representation becomes infeasible. The Deep Q-Network (DQN) replaces the table with a neural network parameterized by weights , trained by minimising the Bellman residual loss:
where is a periodically updated target network that stabilises training by decoupling the moving target from the network being updated. A replay buffer stores historical transitions , and mini-batches are drawn uniformly at random to break temporal correlations in the training data.
Ning et al. (2021) apply a Double DQN architecture — which reduces Q-value overestimation by separating action selection from value evaluation — to execution with limit and market orders. Their model is trained on a calibrated synthetic LOB and evaluated against IS and VWAP benchmarks, showing that the learned policy adapts placement depth to real-time LOB features in a way that the static aggressiveness matrix cannot.

Figure 3:Training a DQN execution tactic. Left: training curve — mean episodic reward per episode over 2,000 episodes (smoothed with a 50-episode rolling average). The agent progresses from random exploration to approximately optimal policy. Right: learned Q-values as a function of remaining time and remaining fraction for the action (best ask placement), showing that the DQN has learned a representation qualitatively consistent with the theoretical aggressiveness matrix: higher Q-values (greener) at early times and low remaining fractions, lower Q-values (redder) at late times with large remaining fractions where more aggressive actions dominate.
Policy gradient methods¶
Q-learning operates in a discretized action space. When the placement depth is treated as a continuous variable — or when the policy must be stochastic to handle exploration and variability in LOB conditions — policy gradient methods are more appropriate. The Proximal Policy Optimisation (PPO) algorithm Sutton & Barto, 2018 maintains a parameterized policy (e.g., a Gaussian over continuous depths) and updates it by ascending a clipped surrogate objective that prevents destabilising large policy updates:
where is the probability ratio and is the estimated advantage function.
Actor-critic methods (A2C, A3C) combine a policy network (actor) with a value network (critic) that estimates the expected return from each state. The critic reduces the variance of policy gradient estimates by providing a baseline for the advantage function. This structure mirrors the relationship between the optimal control value function and the optimal policy : the critic approximates and the actor approximates .
Recent developments in RL for execution have explored several directions. Reward shaping augments the terminal execution cost with intermediate milestones (e.g., rewarding the agent for maintaining pace relative to a VWAP schedule). Multi-agent frameworks model the interaction between the execution tactic and other market participants — momentum traders, market makers — explicitly, training the tactic in a game-theoretic rather than single-agent setting. Offline RL trains on historical order book data without an explicit simulator, avoiding the model misspecification that can occur when training on a stylised generative model. All of these extensions move in the direction of reducing the gap between the theoretical ideal and robust practical deployment.
Exercises¶
Verify the analytical solution , where , by substituting into the HJB equation and checking that the initial condition is satisfied. Show that as the solution reduces to the risk-neutral form . Interpret the limiting cases and in terms of the optimal depth , and describe how the market-order threshold at depends on .
In the single-market tactic, the optimal placement is where . (a) Show that the optimal placement depth is always weakly smaller when is larger: for all . This is the “urgency is monotone in inventory” property. (b) Show that is weakly decreasing in for fixed — the tactic becomes more aggressive as time runs out.
In the two-venue SOR model with venues indexed , venue 1 is liquid (, ) and venue 2 is illiquid (, ). (a) Compute the optimal depths and as functions of the marginal inventory cost . (b) For (moderate urgency), compute the aggregate fill rate and compare it to the fill rate achievable on venue 1 alone if the same total aggression is concentrated there. (c) Interpret the result: why does the SOR outperform single-venue execution?
Consider a pegged-to-mid sell order, so is held constant (not optimized) but the limit price tracks continuously. Assuming the mid-price follows an arithmetic Brownian motion with volatility and no drift, and that fills occur when a market buy order arrives (at rate per unit time, independently of the price level), show that the expected proceeds of the pegged order relative to the mid-price at the time of posting are equal to for any . Why does price volatility not affect expected proceeds in this case? Under what market microstructure assumptions does this result break down?
Consider the RL formulation of the tactic. (a) Show that the tabular Q-learning algorithm converges to the optimal Q-function satisfying the Bellman equation under the usual conditions (sufficient exploration, diminishing step sizes). (b) Explain why the function approximation version (DQN) does not generally converge to the exact Bellman solution, even with infinite data, and describe the two main stabilisation techniques (experience replay and target network) used to make the algorithm work in practice.
Iceberg order trade-off. A sell order of shares is to be executed passively over minutes. Compare two strategies: (i) a fully visible limit order at depth tick, filled at rate fills/min; (ii) an iceberg order with tip shares, where each tip fills at the same rate fills/min but the tactic must replenish the tip after each fill, incurring a 5-second queue re-entry delay. Compute the expected number of units filled in 30 minutes under each strategy, assuming fills are independent Poisson events. What tip size minimises the total expected time to complete the order?
- Avellaneda, M., & Stoikov, S. (2008). High-frequency trading in a limit order book. Quantitative Finance, 8(3), 217–224. 10.1080/14697680701381228
- Cont, R., Stoikov, S., & Talreja, R. (2010). A Stochastic Model for Order Book Dynamics. Operations Research, 58(3), 549–563. 10.1287/opre.1090.0780
- Cartea, Á., & Jaimungal, S. (2015). Optimal Execution with Limit and Market Orders. Quantitative Finance, 15(8), 1279–1291. 10.1080/14697688.2015.1032543
- Ganchev, K., Nevmyvaka, Y., Kearns, M., & Wortman Vaughan, J. (2010). Censored Exploration and the Dark Pool Problem. Communications of the ACM, 53(5), 99–107. 10.1145/1735223.1735248
- Cartea, Á., Jaimungal, S., & Penalva, J. (2015). Algorithmic and High-Frequency Trading. Cambridge University Press.
- Nevmyvaka, Y., Feng, Y., & Kearns, M. (2006). Reinforcement Learning for Optimized Trade Execution. Proceedings of the 23rd International Conference on Machine Learning (ICML), 673–680.
- Ning, B., Lin, F. H. T., & Jaimungal, S. (2021). Double Deep Q-Network for Optimal Execution Strategies. Applied Mathematical Finance, 28(2), 109–139. 10.1080/1350486X.2021.1980553
- Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press.