Sharing training compute

All nodes can share gradients with the whole network via gossip. With that, the only remaining problems are (1) making sure honest nodes are only working on a narrow beam of models, and (2) poisoning. I think both can be solved. Some background:

• Both problems can be solved with an open-network BFT consensus algorithm.
• There are two known safe classes of open-network BFT consensus algorithms: quorum intersection (proof-of-work, proof-of-stake) and phase change.
• Phase change consensus algorithms, e.g., the Avalanche protocol, can work with subjective evaluations.

If you throw out the privacy requirements (i.e., don't let people provide uninspectable data), you can:

1. Have each person manually check data to decide if they think it should be in the model, and
2. Randomly check to make sure gradients were likely calculated from the provided data.

The manual checks don't have to be fast or thorough, just enough that someone on the network will identify bad data and report it for others to verify before the gradients get too stale. Before the gradients go stale, you can subtract them out if you inadvertently added them in at some point.

For mostly-trusted parties that provide huge amounts of data (e.g., companies providing billions of datapoints), you can randomly sample the data to verify it with 99.99% certainty. For unknown parties that provide little data, you can verify all of the data. The network's verification bandwidth should be proportional to the number of participants, so it should balance out.