Feature: Transaction Solidifier

[luca-moser]

Description

In order to have a relevant view of the graph from a given depth towards the tips, we want to be able to mark transactions as solid from bottom->top. In a scenario where the TipSolidifier is disabled, only the TransactionValidator will mark transactions as solid (besides the MilestoneSolidifer), by performing once a quick solid check on approvers of transactions which were set to be solid up on arrival. The TransactionsValidator's quick solidification thread runs every 750 milliseconds.

This means that if we are receiving transactions out of order and a given transaction was missed by the TransactionValidator quick solidification thread, the transaction will only be marked as solid once a new milestone is issued (because it triggers a solidification of all approved transactions).

This PR adds a separate component called TransactionSolidifier with its implementation QuickTransactionSolidifier which starts at a configurable depth (default: 3) and walks from the given milestone up the graph towards the tips traversing all transaction and performing a quick solid check on each of them. The QuickTransactionSolidifier runs every 10 seconds (configurable).

Testing this component on the ICC network, I wanted to see how long such mechanism would take. Luckily, the operation seems to be very quick as shown by the given log outputs:

06/05 17:49:30.697 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 19 and traversed 1321 transactions, took 45 ms
06/05 17:49:33.357 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251301
06/05 17:49:40.763 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 21 and traversed 1385 transactions, took 65 ms
06/05 17:49:43.358 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251365
06/05 17:49:50.846 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 9 and traversed 1439 transactions, took 77 ms
06/05 17:49:53.360 [Neighbor Router] INFO  NeighborRouter:551 - establishing connections to 1 wanted neighbors [tcp://node03.x-vps.com:15600]
06/05 17:49:53.360 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251413
06/05 17:50:00.898 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 7 and traversed 1490 transactions, took 51 ms
06/05 17:50:03.361 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251461
06/05 17:50:04.456 [Neighbor Router] INFO  NeighborRouter:275 - couldn't establish connection to neighbor 116.203.156.13:15600, will attempt to reconnect later. reason: Connection timed out
06/05 17:50:10.962 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 21 and traversed 1536 transactions, took 63 ms
06/05 17:50:13.363 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251507
06/05 17:50:21.023 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 17 and traversed 1582 transactions, took 60 ms
06/05 17:50:23.366 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251551
06/05 17:50:31.078 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 9 and traversed 1624 transactions, took 54 ms

Note that updated N means that the N transactions were updated to be solid and traversed N means how many transactions were traversed while walking up the graph towards the tips. Note also that updated N shows that the TransactionValidator quick solidification thread misses some transactions when it runs which are however always going to be catched by the QuickTransactionSolidifier.

This component doesn't appear even in YourKit's profiling because it's very cheap.

Fixes # (issue)
#1009
#1013
#1011 (maybe?)

The solidifier in TransactionValidator can be removed through #1485.

Type of change

• Enhancement (a non-breaking change which adds functionality)

How Has This Been Tested?

On the ICC network

Checklist:

Please delete items that are not relevant.

• My code follows the style guidelines for this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• New and existing unit tests pass locally with my changes