🧬 FIFO Lot Engine

The package backend/app/services/financial_math/fifo/ contains LibreFolio's pure, event-sourced FIFO engine for one asset across many brokers. Unlike wac.md, which computes an aggregated per-position weighted average cost, FifoLotEngine keeps individual lots alive through buys, sells, splits, transfers, and adjustments so the "FIFO Lots Analysis" panel can explain which lot moved where, when, and why.

The package belongs to the financial math layer and re-exports nothing: import each name from the module that defines it — the data types (input transactions, events, lots, fragments, closures, economic allocations, FifoEngineResult) from models.py, eligible_income_quantity() from eligibility.py, FifoLotEngine and run_fifo_lot_engine() from engine.py.

Use this vs WAC

FifoLotEngine answers lot-level lifecycle questions: FIFO matching, realized P&L per lot, custody fragments, transfer transit, split-adjusted quantities, and lot history for charts/modals.

portfolio_service.compute_wac_iterative() plus services/financial_math/average_cost.py answer position-level cost basis questions: one running WAC per (broker, asset) scope, suitable for transaction validation and broker summaries.

🏗️ Architecture

The engine is intentionally isolated from I/O:

No DB queries
No FX conversion
No quote_base_quantity scaling
No current-price fetches
No asset price-feed dependency

It consumes already-loaded transactions plus a few deterministic helpers, then returns a FifoEngineResult. FIFO cost matching and realized P&L always come from real transaction prices (amount / quantity, cost-basis overrides, and split/transfer replay), never from the asset price feed. Open-position valuation is added later by LotsAnalysisService through the unified build_asset_price_series() resolver.

def run_fifo_lot_engine(
    transactions: Sequence[TransactionLike | FifoInputTransaction],
    broker_shorting: dict[int, bool],
    *,
    split_ratios_by_tx_id: dict[int, Decimal] | None = None,
    split_event_ids_by_tx_id: dict[int, int] | None = None,
    reference_price_lookup: ReferencePriceLookup | None = None,
    economic_events: Sequence[EconomicEvent] = (),
    target_currency: str = "",
) -> FifoEngineResult:

⚙️ Inputs

Input	Meaning
`transactions`	Chronological asset transactions for one asset only.
`broker_shorting`	Per-broker flag controlling whether a `SELL` remainder may open a `SHORT` lot.
`split_ratios_by_tx_id`	Split transaction ID → ratio used to synthesize `SPLIT` events.
`split_event_ids_by_tx_id`	Split transaction ID → its `SPLIT` `AssetEvent` id, carried by the synthesized event as `asset_event_id`. Yield on Cost passes it; the lots analysis does not.
`reference_price_lookup`	Optional pure lookup used by current implementation when `ADJUSTMENT_IN` opens a new LONG lot.
`economic_events`	Asset-linked `DIVIDEND` / `INTEREST` / `FEE` / `TAX` as `EconomicEvent`s, each with its native amount and the target-currency amount its caller resolved; consumed by the economic allocation stage.
`target_currency`	The caller's report currency; each `EconomicEvent` also carries its own `target_currency`.

📤 Output

FifoEngineResult contains:

normalized classified_events
final lots
custody fragment_intervals
FIFO closures
issues
derived analysis_status (COMPLETE, DEGRADED, or FAILED)

The engine does not compute market valuation

FifoEngineResult intentionally exposes no market-valuation helper. Market value (open_quantity / quote_base_quantity * market_price), target-currency FX, and estimated-at-cost fallback are presentation concerns owned by the lots analysis — LotsAnalysisService and its functions in financial_math/lots/ — not this engine. Keeping them out of the engine avoids a latent ×quote_base_quantity valuation bug.

🧱 Core Data Structures

📦 `FifoLot`

One FIFO lot. Usually opened by a BUY, by the remainder of an ADJUSTMENT_IN, or by a short-opening SELL.

Field	Meaning
`lot_id`	Stable lot identifier. In practice equals opening transaction ID.
`asset_id`	Asset handled by this engine instance.
`direction`	`"LONG"` or `"SHORT"`.
`opening_transaction_id`	Source transaction that opened lot.
`opening_broker_id`	Broker where lot started.
`opening_date`	Opening date.
`original_quantity`	Original lot quantity, adjusted later by splits.
`opening_unit_price`	Per-unit opening cost currently attached to lot.
`original_cost`	Original economic cost basis kept invariant across splits/transfers.
`currency`	Transaction currency captured at open time.
`open_quantity`	Remaining open quantity after FIFO closures.
`realized_quantity`	Quantity already closed.
`realized_pnl`	Cumulative realized P&L from closures.
`cumulative_proceeds`	Cumulative sale proceeds for LONG lots, or opening proceeds for SHORT lots.
`reference_unit_price`	Optional reference price the engine resolves for a lot (e.g. when an `ADJUSTMENT_IN` opens a new LONG lot), consumed downstream by `LotsAnalysisService`.
`reference_price_source`	`"exact"`, `"fallback"`, `"unavailable"`, or `None`.

reference_unit_price is stored on the market/quote_base_quantity axis when the resolver supplies one. If no opening quote is available, opening_reference_price() (financial_math/lots/prices.py) falls back to lot.opening_unit_price * quote_base_quantity before the lots analysis computes relative_return; the engine itself does not know quote_base_quantity.

🧩 `FragmentInterval`

Custody fragment for one lot. This is what powers Gantt lanes and custody history.

Field	Meaning
`fragment_id`	Stable fragment key such as origin or transfer-derived IDs.
`lot_id`	Owning lot.
`direction`	Same lot direction (`"LONG"` / `"SHORT"`).
`custody_type`	`"BROKER"` or `"IN_TRANSIT"`.
`quantity`	Quantity living in this fragment interval.
`unit_price`	Cost per unit carried by this fragment.
`start_date`	Inclusive start date.
`broker_id`	Broker for broker-custodied fragments.
`end_date`	`None` while fragment is active.
`source_broker_id`	Transfer source broker for transit fragments.
`destination_broker_id`	Transfer destination broker for transit fragments.

🔚 `LotClosure`

Single FIFO close operation against one fragment.

Field	Meaning
`lot_id`	Closed lot.
`transaction_id`	Transaction causing closure.
`quantity`	Quantity matched in this closure step.
`close_date`	Closure date.
`close_reason`	`"SELL"`, `"BUY"`, or `"ADJUSTMENT_OUT"`.
`fragment_id`	Exact fragment consumed.
`open_unit_price`	Cost carried by consumed fragment.
`close_unit_price`	Sell/buy/adjustment close price.
`realized_pnl`	Realized P&L for this matched piece.
`proceeds`	Non-zero only for LONG `SELL` closures.

📊 `FifoEngineResult`

Returned snapshot of complete run.

Field	Meaning
`asset_id`	Asset processed by engine.
`classified_events`	Normalized event stream after transfer/split classification.
`lots`	Final lots sorted by `(opening_date, lot_id)`.
`fragment_intervals`	All custody intervals sorted by start date.
`closures`	All FIFO closures sorted by close date.
`issues`	Data-quality / unsupported-scenario issues.
`economic_allocation_groups`	3-level economic audit (income + FEE/TAX) with source tx, rule, weights, native+target amounts.
`economic_accumulators_by_lot`	Per-lot `gross_income`, `allocated_fees`, `allocated_taxes` (target currency, positive magnitudes).
`asset_orphan_income` / `asset_orphan_fees` / `asset_orphan_taxes`	Amounts with no eligible lot, kept at asset level.

Useful helpers on result:

analysis_status: "FAILED" if any issue is a non-isolable quantitative-replay error (see _QUANTITATIVE_FAILURE_CODES in models.py), else "DEGRADED" if issues is non-empty, else "COMPLETE"
get_lot_states(lot_id): derives LONG/SHORT, OPEN/PARTIALLY_CLOSED/CLOSED, plus IN_TRANSIT, DISTRIBUTED, DEGRADED
active_fragments(...): filter live custody fragments

🧠 `FifoLotEngine`

Mutable runner around pure input/output contract.

Member	Purpose
`__init__(...)`	Validates non-empty single-asset input and stores runtime config.
`classify_events()`	Normalizes raw transactions into `FifoEvent` objects.
`run()`	Applies each event and emits `FifoEngineResult`.
`signed_quantity_for_broker()`	Current signed broker exposure from active broker fragments.

run() dispatches strictly by normalized event kind:

for event in events:
    if event.kind == "BUY":
        self._apply_buy(event)
    elif event.kind == "SELL":
        self._apply_sell(event)
    elif event.kind == "ADJUSTMENT_IN":
        self._apply_adjustment_in(event)
    # ... TRANSFER_DEPART / TRANSFER_ARRIVE / SPLIT

🔁 Event Handling

EventKind is defined exactly as:

EventKind = Literal[
    "BUY",
    "SELL",
    "ADJUSTMENT_IN",
    "ADJUSTMENT_OUT",
    "SPLIT",
    "TRANSFER_DEPART",
    "TRANSFER_ARRIVE",
]

🗂️ Classification rules

BUY transaction → BUY
SELL transaction → SELL
positive ADJUSTMENT → ADJUSTMENT_IN
negative ADJUSTMENT → ADJUSTMENT_OUT
transaction ID found in split_ratios_by_tx_id → SPLIT
paired TRANSFER legs → synthesized TRANSFER_DEPART + TRANSFER_ARRIVE

📋 Inventory effects

Event kind	Effect on inventory
`BUY`	First closes existing `SHORT` fragments on same broker (`close_reason="BUY"`). Any remainder opens new `LONG` lot.
`SELL`	Closes `LONG` fragments FIFO on same broker. Any remainder opens new `SHORT` lot only if `broker_shorting[broker_id]` is true; otherwise emits `FIFO_SOURCE_QUANTITY_MISSING`.
`ADJUSTMENT_IN`	First closes existing `SHORT` fragments with `close_unit_price = 0`. Any remainder opens zero-price `LONG` lot.
`ADJUSTMENT_OUT`	Closes `LONG` fragments FIFO with `close_unit_price = 0`. If quantity still missing, engine emits issue; it never opens/consumes unsupported SHORT adjustment legs.
`TRANSFER_DEPART`	Extracts `LONG` broker fragments FIFO from source broker, optionally opens `IN_TRANSIT` fragments, records pending transfer pieces, realizes no P&L.
`TRANSFER_ARRIVE`	Closes matching transit fragments and reopens broker custody at destination with same quantity and unit price.
`SPLIT`	Multiplies active fragment quantities by ratio, divides unit prices by ratio, then recomputes lot-level quantities/reference price while preserving cost.

Event ordering on same date

classify_events() sorts same-day events as TRANSFER_DEPART → TRANSFER_ARRIVE → SPLIT → ordinary transactions. This preserves half-open transfer intervals and lets same-day splits see post-transfer custody state.

⛏️ FIFO Matching Algorithm

FIFO is enforced by _broker_fragments(), which sorts active broker fragments by lot age:

return sorted(
    [fragment for fragment in self._active_fragments.values() if ...],
    key=lambda fragment: (
        self._lots[fragment.lot_id].opening_date,
        fragment.lot_id,
        fragment.start_date,
        fragment.fragment_id,
    ),
)

Then _consume_broker_fragments() walks oldest fragments first:

Select matching broker fragments for one direction
Take matched = min(remaining, fragment.quantity)
Call _close_position_piece(...)
Reduce fragment or close it entirely
Continue until requested quantity is satisfied or inventory runs out

🧮 Realized P&L math

LONG close: matched_quantity * (close_unit_price - fragment.unit_price)
SHORT close: matched_quantity * (fragment.unit_price - close_unit_price)

SELL creates proceeds only when closing LONG inventory. BUY closes SHORT inventory but does not add proceeds.

✂️ Split Handling

_apply_split() transforms active fragments in split scope, not historical intervals.

🎯 Split scope

Current judgement call in code:

broker fragments match when fragment.broker_id == split broker
in-transit fragments match when source_broker_id == split broker or destination_broker_id == split broker

🔄 Ratio transformation

For each impacted fragment:

new_quantity = fragment.quantity * ratio
new_unit_price = fragment.unit_price / ratio
fragment cost must remain invariant

After fragment transitions, engine updates each impacted lot:

recompute open_quantity
adjust original_quantity
recompute opening_unit_price = original_cost / original_quantity
scale reference_unit_price by old_open_qty / new_open_qty when available

Split invariant uses tolerance, not exact equality

_COST_INVARIANT_TOLERANCE = Decimal("0.01") exists because ratios like 3:1 produce non-terminating decimals under default Decimal precision. Sub-cent drift from truncation is tolerated; larger drift raises AssertionError.

🚚 Transfer Handling

Transfers are modeled as custody moves, not disposals.

🔗 Pair normalization

classify_events() accepts only bidirectional TRANSFER pairs where:

both rows are TRANSFER
each row points to other via related_transaction_id
one leg quantity is negative, other positive
absolute quantities match
both legs belong to same asset

Otherwise engine records TRANSFER_PAIR_MISSING and skips pair.

🛫 Depart

TRANSFER_DEPART:

refuses current SHORT exposure on source broker (SHORT_TRANSFER_NOT_SUPPORTED)
extracts source LONG fragments FIFO
shrinks/closes source broker fragments
opens IN_TRANSIT fragment when transit_start < transit_end
stores _PendingTransferPiece until arrival

🛬 Arrive

TRANSFER_ARRIVE:

pops pending pieces for pair_id
closes transit fragment on arrival date if one exists
opens destination BROKER fragment with same quantity and unit_price

Lot identity stays same across brokers, so frontend can render one lot life with changing custody lanes.

💸 Economic Allocation Stage

After the replay loop, run() calls _allocate_economics(). This stage is read-only with respect to inventory: quantities, fragments and closures are never mutated. It consumes economic_events (DIVIDEND / INTEREST / FEE / TAX, each carrying a native_amount and a target_amount) and produces per-lot accumulators, an audit trail, and asset-level orphans.

💵 Income (`DIVIDEND` / `INTEREST`)

_allocate_income_pools() groups income by pool key (broker, date, economic_type, native_currency, target_currency) and allocates it to eligible lots:

Eligibility = D-1: a lot is eligible if it has open LONG quantity as of date - 1 (eligible_income_quantity(...), eligibility.py), scoped to the paying broker (including quantity that left that broker as IN_TRANSIT). A BUY made on the income date is therefore not eligible; a lot already closed by end of date - 1 is not either. It is the shared D-1 eligibility seam: transaction-ledger Yield on Cost (backend/app/services/financial_math/yield_on_cost.py) counts eligible quantity with the same public function.
weight w_i = EligibleQty_i / Σ EligibleQty_j, distributed with a running remainder so the pool total is conserved exactly;
no eligible lot → the whole pool becomes asset_orphan_income, with an ASSET_INCOME_NO_ELIGIBLE_LOTS issue and an audit group tagged ASSET_INCOME_HOLDINGS.

💸 FEE / TAX

_allocate_cost_pools() pools asset-linked cost and matches it to operations with a deterministic ladder (_match_cost_operations), weighting by target_amount:

Cost	Matching order (first non-empty wins)
`FEE`	same-day trades → previous-day trades → holdings fallback → orphan
`TAX`	same-day income → same-day trades → previous-day income → previous-day trades → holdings fallback → orphan

The chosen rule is recorded per group (SAME_DAY_MIXED_TRADES, SAME_DAY_TRADES, PREVIOUS_DAY_TRADES, OPEN_LOTS_FALLBACK, income-linked variants). Allocation to a matched trade crosses to the lots that trade touched (BUY → the opened lot; SELL → the FIFO-consumed lots), so a cost follows the same lots the FIFO algorithm already selected. A cost with no eligible target becomes asset_orphan_fees / asset_orphan_taxes.

Per-pool conservation (locked by tests)

For every pool: Σ allocated_to_lots + orphan == converted pool total. TestEconomicConservation asserts this invariant, so allocated FEE/TAX/income can never be silently lost or double-counted.

🔎 Three-level audit

Each pool emits an EconomicAllocationGroup (level 1) → operation_allocations (level 2, one per matched BUY/SELL/income) → lot_allocations (level 3, one per lot), all carrying source_transaction_ids, rule, weight, and both native_* and target_* amounts. The service maps these to economic_allocation_groups for the response; the frontend does not yet render the provenance (only the numeric net breakdown).

Costs without an asset are out of scope here

FEE / TAX with asset_id = null are excluded from economic_events entirely — the Portfolio Engine accounts for them. Only asset-linked cost reaches this stage.

📈 Net annualization handoff

The engine returns realized P&L plus economic accumulators; LotsAnalysisService._build_lot_summaries() derives total_pnl, subtracts allocated FEE/TAX into net_total_pnl, computes net_total_return, and annualizes that net figure over opening_date → closing_date (closed lots) or opening_date → analysis_end (open lots). The shipped LotSummarySchema.annualized_return therefore annualizes net_total_return, not gross total_return.

⚠️ Known Constraints and Gotchas

SHORT support is intentionally partial

SELL may open SHORT lots when broker shorting is enabled, and BUY / positive adjustment may close them. TRANSFER_DEPART on SHORT inventory and ADJUSTMENT_OUT against SHORT inventory are currently rejected with SHORT_TRANSFER_NOT_SUPPORTED and SHORT_ADJUSTMENT_NOT_SUPPORTED.

Reference price behavior is narrower in code than broad system docs may suggest

Current implementation calls _resolve_reference_price() only in _apply_adjustment_in() before opening a remainder LONG lot. Ordinary BUY openings pass reference_resolution=None, so many lots will have reference_unit_price is None unless populated by adjustment flow.

Estimated valuation lives outside FIFO

The engine has no ESTIMATED_AT_COST branch and does not emit CURRENT_PRICE_ASSUMED_AT_COST. When lots analysis has no usable mark for an open LONG lot on a price-less asset, the service values the open slice at cost and sets market_pnl = 0; when the unified resolver can derive a trade-origin mark, that mark is flagged estimated=True on the price-history line.

Issues degrade result instead of aborting run

Missing source quantity, broken transfer pairs, and reference-price gaps are recorded in issues; the replay loop never raises, so the engine still returns best-effort lots/fragments/closures for the rest of the input stream. analysis_status then becomes DEGRADED for isolable (economic) issues, or FAILED for quantity-topology breakages (oversell, broken transfer, short-not-supported) that cannot be isolated because they change which lots later events consume via FIFO order.

🧮 Lots Analysis Service — Service layer that adds FX, quote_base_quantity, income allocation, and DTO building on top of engine output
⚖️ WAC & Cost Basis — Complementary per-position cost-basis engine
🧮 Financial Math Layer — The layer this package belongs to: rules, package map, guards
🧬 FIFO Engine Theory — Theoretical mirror of this page: lot lifecycle, matching, splits, transfers in financial terms
📖 FIFO Lot Analysis Theory — Financial meaning of lot-level FIFO analysis