If you buy a wildly popular stock like Apple or Microsoft, you know exactly what the price is. There are so many buyers and sellers that the gap between what you pay and what you sell for, the bid-ask spread, is tiny. It is often just a fraction of a penny.

But what happens when you step away from the giants? When you trade a small pharmaceutical company or a niche manufacturer that doesn't trade often, that gap widens. It becomes an invisible tax on your investment.

For years, investors and researchers have faced a major problem: for thousands of "illiquid" stocks, we don't actually know what that gap is. We have to guess. And when pension funds or asset managers guess wrong, they lose money, millions of dollars lost simply because they couldn't accurately estimate the cost of entering or exiting a position.

There is a new way to solve this, and it doesn't require a supercomputer. It requires a change in perspective: looking at how small stocks behave in the shadow of their larger counterparts.

The Limitation of Current Estimates

To understand the solution, we first need to look at why the current standard falls short.

The past years, one of the industry standards for estimating spreads has been the Corwin-Schultz (CS) estimator. It is popular because it is elegant. It doesn't require expensive, second-by-second trade data. It just looks at the daily High and Low prices of a stock.

The logic is simple:

If a stock has a huge gap between the High and Low price for the day, the spread is likely wide.
If the High and Low are close together, the spread is likely tight.

The problem is that it is often wrong. The CS estimator assumes that every stock lives on its own island, independent of the rest of the market. It fails to account for the fact that when liquidity dries up for one medical stock, it usually dries up for all medical stocks at the same time.

For illiquid stocks, the exact stocks where you need this data the most, the CS estimator has a systematic bias. It consistently misses the mark.

A New Approach: Reference Stocks

The core idea of my research is surprisingly intuitive. It suggests that we stop trying to analyze illiquid stocks in isolation.

Instead, we should look for a "Reference Stock," a highly liquid peer in the same sector.

The Target: The small, illiquid stock you want to trade (e.g., a small biotech firm).
The Reference: A massive, liquid stock in the same industry (e.g., Pfizer or Merck) where we have perfect data.

Because these two companies are in the same sector, they react to the same macroeconomic weather. If the market gets jittery about healthcare regulation, both stocks will see their spreads widen.

My strategy is to measure the error in the spread estimate of the Reference Stock (where we know the truth) and use that to correct the estimate for the smaller stock.

Understanding Pairwise Beta

Here is where the nuance comes in. You cannot simply copy the error from Pfizer and paste it onto the small biotech firm.

Why? Because the small firm is likely much more volatile. If the market sneezes, Pfizer might dip 1%, but the small biotech might crash 5%. The correction needs to be scaled up to match that volatility.

I introduce a metric called Pairwise Beta.

Pairwise Beta is not the stock's sensitivity to the whole market. It is specifically the sensitivity of the Target Stock to the Reference Stock.

The logic follows a power law:

Adjustment = (Reference Bias) ^ Beta

If the pairwise beta is 1, the stocks move in lockstep, and the adjustment is direct. If the beta is 2 (the target is twice as sensitive), the adjustment is amplified. This captures the reality that riskier, more volatile stocks usually suffer from much worse liquidity droughts.

Why This Approach is Rigorous

If you work with data, you might be skeptical. Borrowing data from another stock sounds risky. My research addresses this with a rigorous pipeline that separates this method from a simple guess.

1. Establishing Ground Truth

To calibrate the model, I first had to calculate the "true" spread from intraday data. This is difficult because you have to determine if a trade was initiated by a buyer or a seller.

The old standard method is only about 79% accurate. I used the Lee-Ready algorithm, which uses quote timing. By aligning trades with the standing quotes, accuracy jumps to 93%.

2. Handling Market Correlations

Standard regression fails here because spreads are dependent on each other; they all move together during market shocks. I use a Dynamic Common Correlated Effects (DCCE) framework.

This involves adding cross-sectional averages to the model. Essentially, the model includes variables that account for the invisible tides that lift or lower all boats in the sector simultaneously, ensuring the beta estimate isn't just picking up noise.

3. Preventing Overfitting

The model considers many variables, such as volatility, turnover, and spread levels. To avoid overfitting, I use Elastic Net regression on rolling windows. This automatically selects only the variables that actually matter right now, allowing the model to adapt as market dynamics shift over time.

Real World Impact

This isn't just an academic exercise. It solves three very expensive problems:

Portfolio Management: Imagine you need to sell 100,000 shares of a small company. If your spread estimate is off by just a few cents, you might underestimate your transaction costs significantly. This method provides a sharper price tag before you trade.
Risk Management: Banks need to know how quickly they can liquidate assets in a crisis. If they use the old method, they might think their inventory is more liquid than it actually is. This beta-adjusted method reveals the true liquidity risk hidden in the portfolio.
Better Research: Academics and analysts often exclude small stocks from studies because the data is too noisy. By cleaning up the noise, we can finally analyze the full market, not just the top 10% of companies.

Limitations

No model is perfect. There are caveats to this approach:

The Peer Problem: It relies on finding a good Reference Stock. If you are trading a truly unique company with no peers, this method struggles.
Market Crises: The relationship between the big and small stock is assumed to be somewhat stable. In a total market collapse, these correlations can break down, making the estimate less reliable exactly when you need it most.

Conclusion

For too long, the financial world accepted that liquidity estimates for small stocks were "good enough." They weren't.

By using Pairwise Beta, we can bridge the gap between the data-rich world of blue-chip stocks and the data-poor world of illiquid assets. It is a method that respects the complexity of the market, acknowledging that stocks move together, while remaining simple enough to implement with standard daily data.