Enhancements for tpu-client-next

[KirillLykov]

Problem

Tpu-client-next is introduced in #2905
This issue is to track features requests.

Proposed by me:

• Add fanout to send to next 2 leaders
• Add option to send with several identities to increase used bandwidth.

Proposed by @godmodegalactus:

• Order transactions to send according to their priority fees. Currently, priority fee is not taken into consideration. This might be an interesting feature for users.
• Use timestamp per transaction, not per batch. Because typically transactions are not generated in batches at one moment in time as you do in the bmk code.
• Change mechanism of age check for transactions

Proposed by Alesssandro:

• ConnectionWorker:run should return Result.

with the API as it is now, it's impossible to tell whether ConnectionWorker::run()
succeeded or failed (other than looking at stats, which is not a good API).
I think that run() should probably return Result, and whenever there's a fatal
un-retriable error it should return the actual error.

• Improve connect to leaders

The way you connect to the leaders can be
improved. I think as part of setup you should connect to all the leaders, then
this case !workers.contains(current_leader) or
worker_for_leader.is_disconnected() should print an error!() - it should never
happen outside of initial setup, and if it happens something is seriously borked
and is going to impact perf, so should be visible in logs

• Shutdown in background task:

I think you should shutdown in a background task, so as not to impact the send
path at all. But these stats currently preclude you from doing so, since you
need to modify self.send_stats_per_addr. Maybe these stats should just be held
in the background task, and you send them over to the scheduler or something.

• Are Quic client parameters optimal?

Having max idle only 2 * KEEP_ALIVE strikes me as wrong in case there's packet
loss. This is something we should measure.