Bitcoin Core
27.99.0
P2P Digital Currency

The BlockPolicyEstimator is used for estimating the feerate needed for a transaction to be included in a block within a certain number of blocks. More...
#include <fees.h>
Classes  
struct  TxStatsInfo 
Public Member Functions  
CBlockPolicyEstimator (const fs::path &estimation_filepath, const bool read_stale_estimates)  
Create new BlockPolicyEstimator and initialize stats tracking classes with default values. More...  
virtual  ~CBlockPolicyEstimator () 
void  processBlock (const std::vector< RemovedMempoolTransactionInfo > &txs_removed_for_block, unsigned int nBlockHeight) EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Process all the transactions that have been included in a block. More...  
void  processTransaction (const NewMempoolTransactionInfo &tx) EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Process a transaction accepted to the mempool. More...  
bool  removeTx (uint256 hash) EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Remove a transaction from the mempool tracking stats for non BLOCK removal reasons. More...  
CFeeRate  estimateFee (int confTarget) const EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
DEPRECATED. More...  
CFeeRate  estimateSmartFee (int confTarget, FeeCalculation *feeCalc, bool conservative) const EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Estimate feerate needed to get be included in a block within confTarget blocks. More...  
CFeeRate  estimateRawFee (int confTarget, double successThreshold, FeeEstimateHorizon horizon, EstimationResult *result=nullptr) const EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Return a specific fee estimate calculation with a given success threshold and time horizon, and optionally return detailed data about calculation. More...  
bool  Write (AutoFile &fileout) const EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Write estimation data to a file. More...  
bool  Read (AutoFile &filein) EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Read estimation data from a file. More...  
void  FlushUnconfirmed () EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Empty mempool transactions on shutdown to record failure to confirm for txs still in mempool. More...  
unsigned int  HighestTargetTracked (FeeEstimateHorizon horizon) const EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Calculation of highest target that estimates are tracked for. More...  
void  Flush () EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Drop still unconfirmed transactions and record current estimations, if the fee estimation file is present. More...  
void  FlushFeeEstimates () EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Record current fee estimations. More...  
std::chrono::hours  GetFeeEstimatorFileAge () 
Calculates the age of the file, since last modified. More...  
Protected Member Functions  
void  TransactionAddedToMempool (const NewMempoolTransactionInfo &tx, uint64_t) override EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Overridden from CValidationInterface. More...  
void  TransactionRemovedFromMempool (const CTransactionRef &tx, MemPoolRemovalReason, uint64_t) override EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Notifies listeners of a transaction leaving mempool. More...  
void  MempoolTransactionsRemovedForBlock (const std::vector< RemovedMempoolTransactionInfo > &txs_removed_for_block, unsigned int nBlockHeight) override EXCLUSIVE_LOCKS_REQUIRED(!m_cs_fee_estimator) 
Protected Member Functions inherited from CValidationInterface  
~CValidationInterface ()=default  
Protected destructor so that instances can only be deleted by derived classes. More...  
virtual void  UpdatedBlockTip (const CBlockIndex *pindexNew, const CBlockIndex *pindexFork, bool fInitialDownload) 
Notifies listeners when the block chain tip advances. More...  
virtual void  BlockConnected (ChainstateRole role, const std::shared_ptr< const CBlock > &block, const CBlockIndex *pindex) 
Notifies listeners of a block being connected. More...  
virtual void  BlockDisconnected (const std::shared_ptr< const CBlock > &block, const CBlockIndex *pindex) 
Notifies listeners of a block being disconnected Provides the block that was disconnected. More...  
virtual void  ChainStateFlushed (ChainstateRole role, const CBlockLocator &locator) 
Notifies listeners of the new active block chain ondisk. More...  
virtual void  BlockChecked (const CBlock &, const BlockValidationState &) 
Notifies listeners of a block validation result. More...  
virtual void  NewPoWValidBlock (const CBlockIndex *pindex, const std::shared_ptr< const CBlock > &block) 
Notifies listeners that a block which builds directly on our current tip has been received and connected to the headers tree, though not validated yet. More...  
Private Member Functions  
unsigned int nBestSeenHeight  GUARDED_BY (m_cs_fee_estimator) 
unsigned int firstRecordedHeight  GUARDED_BY (m_cs_fee_estimator) 
unsigned int historicalFirst  GUARDED_BY (m_cs_fee_estimator) 
unsigned int historicalBest  GUARDED_BY (m_cs_fee_estimator) 
std::map< uint256, TxStatsInfo > mapMemPoolTxs  GUARDED_BY (m_cs_fee_estimator) 
std::unique_ptr< TxConfirmStats > feeStats  PT_GUARDED_BY (m_cs_fee_estimator) 
Classes to track historical data on transaction confirmations. More...  
std::unique_ptr< TxConfirmStats > shortStats  PT_GUARDED_BY (m_cs_fee_estimator) 
std::unique_ptr< TxConfirmStats > longStats  PT_GUARDED_BY (m_cs_fee_estimator) 
unsigned int trackedTxs  GUARDED_BY (m_cs_fee_estimator) 
unsigned int untrackedTxs  GUARDED_BY (m_cs_fee_estimator) 
std::vector< double > buckets  GUARDED_BY (m_cs_fee_estimator) 
std::map< double, unsigned int > bucketMap  GUARDED_BY (m_cs_fee_estimator) 
bool  processBlockTx (unsigned int nBlockHeight, const RemovedMempoolTransactionInfo &tx) EXCLUSIVE_LOCKS_REQUIRED(m_cs_fee_estimator) 
Process a transaction confirmed in a block. More...  
double  estimateCombinedFee (unsigned int confTarget, double successThreshold, bool checkShorterHorizon, EstimationResult *result) const EXCLUSIVE_LOCKS_REQUIRED(m_cs_fee_estimator) 
Helper for estimateSmartFee. More...  
double  estimateConservativeFee (unsigned int doubleTarget, EstimationResult *result) const EXCLUSIVE_LOCKS_REQUIRED(m_cs_fee_estimator) 
Helper for estimateSmartFee. More...  
unsigned int  BlockSpan () const EXCLUSIVE_LOCKS_REQUIRED(m_cs_fee_estimator) 
Number of blocks of data recorded while fee estimates have been running. More...  
unsigned int  HistoricalBlockSpan () const EXCLUSIVE_LOCKS_REQUIRED(m_cs_fee_estimator) 
Number of blocks of recorded fee estimate data represented in saved data file. More...  
unsigned int  MaxUsableEstimate () const EXCLUSIVE_LOCKS_REQUIRED(m_cs_fee_estimator) 
Calculation of highest target that reasonable estimate can be provided for. More...  
bool  _removeTx (const uint256 &hash, bool inBlock) EXCLUSIVE_LOCKS_REQUIRED(m_cs_fee_estimator) 
A nonthreadsafe helper for the removeTx function. More...  
Private Attributes  
const fs::path  m_estimation_filepath 
Mutex  m_cs_fee_estimator 
Static Private Attributes  
static constexpr unsigned int  SHORT_BLOCK_PERIODS = 12 
Track confirm delays up to 12 blocks for short horizon. More...  
static constexpr unsigned int  SHORT_SCALE = 1 
static constexpr unsigned int  MED_BLOCK_PERIODS = 24 
Track confirm delays up to 48 blocks for medium horizon. More...  
static constexpr unsigned int  MED_SCALE = 2 
static constexpr unsigned int  LONG_BLOCK_PERIODS = 42 
Track confirm delays up to 1008 blocks for long horizon. More...  
static constexpr unsigned int  LONG_SCALE = 24 
static const unsigned int  OLDEST_ESTIMATE_HISTORY = 6 * 1008 
Historical estimates that are older than this aren't valid. More...  
static constexpr double  SHORT_DECAY = .962 
Decay of .962 is a halflife of 18 blocks or about 3 hours. More...  
static constexpr double  MED_DECAY = .9952 
Decay of .9952 is a halflife of 144 blocks or about 1 day. More...  
static constexpr double  LONG_DECAY = .99931 
Decay of .99931 is a halflife of 1008 blocks or about 1 week. More...  
static constexpr double  HALF_SUCCESS_PCT = .6 
Require greater than 60% of X feerate transactions to be confirmed within Y/2 blocks. More...  
static constexpr double  SUCCESS_PCT = .85 
Require greater than 85% of X feerate transactions to be confirmed within Y blocks. More...  
static constexpr double  DOUBLE_SUCCESS_PCT = .95 
Require greater than 95% of X feerate transactions to be confirmed within 2 * Y blocks. More...  
static constexpr double  SUFFICIENT_FEETXS = 0.1 
Require an avg of 0.1 tx in the combined feerate bucket per block to have stat significance. More...  
static constexpr double  SUFFICIENT_TXS_SHORT = 0.5 
Require an avg of 0.5 tx when using short decay since there are fewer blocks considered. More...  
static constexpr double  MIN_BUCKET_FEERATE = 1000 
Minimum and Maximum values for tracking feerates The MIN_BUCKET_FEERATE should just be set to the lowest reasonable feerate we might ever want to track. More...  
static constexpr double  MAX_BUCKET_FEERATE = 1e7 
static constexpr double  FEE_SPACING = 1.05 
Spacing of FeeRate buckets We have to lump transactions into buckets based on feerate, but we want to be able to give accurate estimates over a large range of potential feerates Therefore it makes sense to exponentially space the buckets. More...  
The BlockPolicyEstimator is used for estimating the feerate needed for a transaction to be included in a block within a certain number of blocks.
At a high level the algorithm works by grouping transactions into buckets based on having similar feerates and then tracking how long it takes transactions in the various buckets to be mined. It operates under the assumption that in general transactions of higher feerate will be included in blocks before transactions of lower feerate. So for example if you wanted to know what feerate you should put on a transaction to be included in a block within the next 5 blocks, you would start by looking at the bucket with the highest feerate transactions and verifying that a sufficiently high percentage of them were confirmed within 5 blocks and then you would look at the next highest feerate bucket, and so on, stopping at the last bucket to pass the test. The average feerate of transactions in this bucket will give you an indication of the lowest feerate you can put on a transaction and still have a sufficiently high chance of being confirmed within your desired 5 blocks.
Here is a brief description of the implementation: When a transaction enters the mempool, we track the height of the block chain at entry. All further calculations are conducted only on this set of "seen" transactions. Whenever a block comes in, we count the number of transactions in each bucket and the total amount of feerate paid in each bucket. Then we calculate how many blocks Y it took each transaction to be mined. We convert from a number of blocks to a number of periods Y' each encompassing "scale" blocks. This is tracked in 3 different data sets each up to a maximum number of periods. Within each data set we have an array of counters in each feerate bucket and we increment all the counters from Y' up to max periods representing that a tx was successfully confirmed in less than or equal to that many periods. We want to save a history of this information, so at any time we have a counter of the total number of transactions that happened in a given feerate bucket and the total number that were confirmed in each of the periods or less for any bucket. We save this history by keeping an exponentially decaying moving average of each one of these stats. This is done for a different decay in each of the 3 data sets to keep relevant data from different time horizons. Furthermore we also keep track of the number unmined (in mempool or left mempool without being included in a block) transactions in each bucket and for how many blocks they have been outstanding and use both of these numbers to increase the number of transactions we've seen in that feerate bucket when calculating an estimate for any number of confirmations below the number of blocks they've been outstanding.
We want to be able to estimate feerates that are needed on tx's to be included in a certain number of blocks. Every time a block is added to the best chain, this class records stats on the transactions included in that block
CBlockPolicyEstimator::CBlockPolicyEstimator  (  const fs::path &  estimation_filepath, 
const bool  read_stale_estimates  
) 

virtualdefault 

private 

private 

private 
Helper for estimateSmartFee.
Return a fee estimate at the required successThreshold from the shortest time horizon which tracks confirmations up to the desired target.
If checkShorterHorizon is requested, also allow short time horizon estimates for a lower target to reduce the given answer
Definition at line 804 of file fees.cpp.

private 
CFeeRate CBlockPolicyEstimator::estimateFee  (  int  confTarget  )  const 
CFeeRate CBlockPolicyEstimator::estimateRawFee  (  int  confTarget, 
double  successThreshold,  
FeeEstimateHorizon  horizon,  
EstimationResult *  result = nullptr 

)  const 
CFeeRate CBlockPolicyEstimator::estimateSmartFee  (  int  confTarget, 
FeeCalculation *  feeCalc,  
bool  conservative  
)  const 
Estimate feerate needed to get be included in a block within confTarget blocks.
estimateSmartFee returns the max of the feerates calculated with a 60% threshold required at target / 2, an 85% threshold required at target and a 95% threshold required at 2 * target.
If no answer can be given at confTarget, return an estimate at the closest target where one can be given. 'conservative' estimates are valid over longer time horizons also.
Each calculation is performed at the shortest time horizon which tracks the required target. Conservative estimates, however, required the 95% threshold at 2 * target be met for any longer time horizons also.
true is passed to estimateCombined fee for target/2 and target so that we check the max confirms for shorter time horizons as well. This is necessary to preserve monotonically increasing estimates. For nonconservative estimates we do the same thing for 2*target, but for conservative estimates we want to skip these shorter horizons checks for 2*target because we are taking the max over all time horizons so we already have monotonically increasing estimates and the purpose of conservative estimates is not to let short term fluctuations lower our estimates by too much.
Definition at line 867 of file fees.cpp.
void CBlockPolicyEstimator::Flush  (  ) 
void CBlockPolicyEstimator::FlushFeeEstimates  (  ) 
void CBlockPolicyEstimator::FlushUnconfirmed  (  ) 
std::chrono::hours CBlockPolicyEstimator::GetFeeEstimatorFileAge  (  ) 

inlineprivate 

inlineprivate 

inlineprivate 

inlineprivate 

private 

inlineprivate 

inlineprivate 

private 

private 
unsigned int CBlockPolicyEstimator::HighestTargetTracked  (  FeeEstimateHorizon  horizon  )  const 

private 

private 

overrideprotectedvirtual 
Reimplemented from CValidationInterface.
Definition at line 587 of file fees.cpp.
void CBlockPolicyEstimator::processBlock  (  const std::vector< RemovedMempoolTransactionInfo > &  txs_removed_for_block, 
unsigned int  nBlockHeight  
) 

private 
void CBlockPolicyEstimator::processTransaction  (  const NewMempoolTransactionInfo &  tx  ) 

private 
Classes to track historical data on transaction confirmations.

private 

private 
bool CBlockPolicyEstimator::Read  (  AutoFile &  filein  ) 
bool CBlockPolicyEstimator::removeTx  (  uint256  hash  ) 

overrideprotectedvirtual 
Overridden from CValidationInterface.
Reimplemented from CValidationInterface.
Definition at line 577 of file fees.cpp.

overrideprotectedvirtual 
Notifies listeners of a transaction leaving mempool.
This notification fires for transactions that are removed from the mempool for the following reasons:
This does not fire for transactions that are removed from the mempool because they have been included in a block. Any client that is interested in transactions removed from the mempool for inclusion in a block can learn about those transactions from the MempoolTransactionsRemovedForBlock notification.
Transactions that are removed from the mempool because they conflict with a transaction in the new block will have TransactionRemovedFromMempool events fired before the BlockConnected event is fired. If multiple blocks are connected in one step, then the ordering could be:
Called on a background thread.
Reimplemented from CValidationInterface.
Definition at line 582 of file fees.cpp.
bool CBlockPolicyEstimator::Write  (  AutoFile &  fileout  )  const 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

mutableprivate 

private 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 
Minimum and Maximum values for tracking feerates The MIN_BUCKET_FEERATE should just be set to the lowest reasonable feerate we might ever want to track.
Historically this has been 1000 since it was inheriting DEFAULT_MIN_RELAY_TX_FEE and changing it is disruptive as it invalidates old estimates files. So leave it at 1000 unless it becomes necessary to lower it, and then lower it substantially.

staticprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 

staticconstexprprivate 