You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Initial peer discovery will be done via static configuration. Those bootstrap/seed nodes will then be used by go-libp2p-discovery to connect to the rest of the network and maintain a peer list.
All those examples are using public libp2p bootstrapping addresses.
There are two three options:
Use public bootstrapping nodes.
Use own bootstrapping nodes (also reachable from the Internet).
One set of seed nodes for all all ORU networks.
One set of seed nodes per every application/network
The more "granular" approach, the more "specific" peers will be returned by seed nodes.
In 2.ii, there is no need to further filter peers based on supported protocol, app, etc.
The 1. should probably result in very large number of peers, but only few will be actually interesting.
And this is not only configuration issue. If we allow 1. or 2.i, we need a filter peers by protocol and application/network.
And this is not only configuration issue. If we allow 1. or 2.i, we need a filter peers by protocol and application/network.
But the filtering will be done by libp2p internally and does not require any additional code from our side, right?
If so, can't we go with 1. and optimize for "granularity" as we progress with the rest of the optimint stack? I think 1. would have the benefit that these nodes are simply available and we can use them directly without bothering on spinning up our own bootstrap nodes.
My experiment above, spawned 100 libp2p hosts, and 1 "seed node" and then used DHT to find connections. After "bootstrapping DHT" (internat DHT thing, do not confuse with seed connections bootstrapping) most hosts know about and are connected to other peers. I tried "active discovery" (DHT based namespace Advertise + FindPeers), hoping to get hosts "grouped by namespace", but in the end everyone is connected to everyone else (with some small exceptions, but I guess it's just a matter of time).
Is it a problem if everyone is connected to everyone though? For LL-core (at least for the MVP) we will accept that our peers connect to all IPFS nodes. Later we will try to narrow it down to ipfs peers that run a particular protocol ID s.t. ll nodes only talk to ll nodes (basically ipfs nodes with a particular plugin installed). ref: https://github.com/lazyledger/lazyledger-core/issues/154
To make nodes prefer connections for applications they are more interested in, use ConnectionMaganer to tag peers with namespace and some weight num the namespace have for the concrete node. ConnectionManager then will drop unneeded connections if the connections limit exceeds while keeping useful(with weight) peers connected.
Does the 2.ii option force application developers to host bootstrap nodes for their applications?
I am leaning towards having one big network(separate from IPFS) that would allow any node to access any namespaced application/overlay network by finding nodes that are advertising themselves as a part of that namespace. That would require us to have only one static set of bootstrap peers for initial bootstrapping where later nodes could bootstrap themselves through a saved dynamic list of more useful peers.
@tzdybal, what is the difference between "seed" and "bootstrap" node?
There is no difference (except the name).
Does the 2.ii option force application developers to host bootstrap nodes for their applications?
Yes, 2.ii is the most simplified version, where each network is absolutely separated, consists of participating nodes only.
Both 1 and 2.i requires the same amount of development - we need to find peers for exact application. Tagging peers to select those interesting, while participating in broader network seems to be very good option. @Wondertan could you elaborate why you're leaning towards network separated from IPFS? Pros I can see: not cluttering IPFS network, smaller number of peers to check/filter.
I think, if we connect to the main IPFS network, peers will be connected to a lot (hundreds of) nodes and most of them will not have the LL plugin installed. Hence it's not really useful to connect to them.
Both 1 and 2.i requires the same amount of development - we need to find peers for exact application.
I was somehow assuming that this is sth that libp2p provides without us having implement the filtering etc our self. Is that not the case?
It's not "automagical" AFAIK, but we can use existing components.
We can go with DHT peer discovery with using Advertise and FindPeers. After finding peers we can connect to them (open a stream) and (as proposed by @Wondertan) tag them in connection manager, to ensure that we stay connected.
This should be the most flexible approach (so 1, 2.i and 2.ii are supported).
@Wondertan could you elaborate why you're leaning towards network separated from IPFS?
DHT protocol is not finished
NAT issues are not solved yet
We introduce a place for side effects to arise while using a public network
All of these may become an issue at the beginning of the journey for our network. I would suggest starting independently to learn how to manage infrastructure ourselves and not to break things for PL, as they are really mad about content resolution timings(e.g. one node with the wrong DHTMode option can affect the whole network). Only after becoming confident about managing the network ourselves, we can consider merging with a public IPFS.
BTW, in the ideal world, it would be better for all networks to speak one DHT protocol(maybe even incentivized) to help everyone searching for needed content. However, we are quite far from that ideal future, though we might get to at some point.
Can you share some pointers about what is missing on the DHT?
NAT issues are not solved yet
What are the NAT issues sepcifically?
BTW, in the ideal world, it would be better for all networks to speak one DHT protocol(maybe even incentivized) to help everyone searching for needed content.
In which way would the DHT protocol be incentivised?
Can you share some pointers about what is missing on the DHT?
It works but there are multiple things missing in the protocol to meet the paper, like this, and there are maaany other practical(not in the paper) things left that are listed as epic issues in the repo. PL was not doing a lot of work on DHT last year as Filecoin does not rely on it that much, as it registers miners addresses onchain for discovery
What are the NAT issues specifically?
Running our nodes behind the NAT and misconfiguring them can degrade the public network. There is an automatic solution, but it still fails to detect proper configuration in all cases. While we can ensure that all our nodes are properly configured, we cannot ensure the same for all the test/incetivised network participants, that is why I am suggesting to be independent at the beginning and merge after being confident that our network works well without any other help from 3rd party. Also, through that time NAT workaround solution should mature as well for us not to think about this issue at all.
In which way would the DHT protocol be incentivised?
IDK exactly :) That last paragraph about the ideal world is not really connected to our issue, that was just thoughts that I missed to format properly.
GossipSub should be sufficient for both gossiping and peering
To achieve 2.1:
seed nodes serves DHT and are used to find peers belonging to particular ORU network,
after establishing connections to ORU-specific nodes, we can rely on GossipSub
Technical notes:
Setup GossipSub before connecting to peers
be careful with setting DHT mode to Server (if node is not reachable (for example behind NAT), it degrades network performance, as the node is advertised, and "failing" connection may take a lot of time).
Activity
tzdybal commentedon Feb 26, 2021
For better UX, we should re-use options/configs already available in cosmos-sdk apps.
(This will also greatly limit changes required in cosmos-sdk.)
tzdybal commentedon Mar 4, 2021
It seems that
go-libp2p-kad-dht, which provides most advanced peer discovery for libp2p, can also handle bootstrapping.tzdybal commentedon Mar 4, 2021
ref: libp2p/go-libp2p#222
liamsi commentedon Mar 4, 2021
There are a few examples using the libp2p bootstrap nodes here: https://github.com/libp2p/go-libp2p-examples/tree/master/
tzdybal commentedon Mar 5, 2021
All those examples are using public libp2p bootstrapping addresses.
There are
twothree options:The more "granular" approach, the more "specific" peers will be returned by seed nodes.
In
2.ii, there is no need to further filter peers based on supported protocol, app, etc.The
1.should probably result in very large number of peers, but only few will be actually interesting.And this is not only configuration issue. If we allow
1.or2.i, we need a filter peers by protocol and application/network.liamsi commentedon Mar 5, 2021
But the filtering will be done by libp2p internally and does not require any additional code from our side, right?
If so, can't we go with
1.and optimize for "granularity" as we progress with the rest of the optimint stack? I think 1. would have the benefit that these nodes are simply available and we can use them directly without bothering on spinning up our own bootstrap nodes.tzdybal commentedon Mar 8, 2021
I was experimenting with it, but it seems that we will need to do some filtering. Now I'm looking at connection gater and connection manager.
My experiment above, spawned 100 libp2p hosts, and 1 "seed node" and then used DHT to find connections. After "bootstrapping DHT" (internat DHT thing, do not confuse with seed connections bootstrapping) most hosts know about and are connected to other peers. I tried "active discovery" (DHT based namespace
Advertise+FindPeers), hoping to get hosts "grouped by namespace", but in the end everyone is connected to everyone else (with some small exceptions, but I guess it's just a matter of time).liamsi commentedon Mar 8, 2021
Is it a problem if everyone is connected to everyone though? For LL-core (at least for the MVP) we will accept that our peers connect to all IPFS nodes. Later we will try to narrow it down to ipfs peers that run a particular protocol ID s.t. ll nodes only talk to ll nodes (basically ipfs nodes with a particular plugin installed). ref: https://github.com/lazyledger/lazyledger-core/issues/154
Wondertan commentedon Mar 8, 2021
@tzdybal, what is the difference between "seed" and "bootstrap" node?
Wondertan commentedon Mar 8, 2021
To make nodes prefer connections for applications they are more interested in, use ConnectionMaganer to tag peers with namespace and some weight num the namespace have for the concrete node. ConnectionManager then will drop unneeded connections if the connections limit exceeds while keeping useful(with weight) peers connected.
Wondertan commentedon Mar 8, 2021
Does the 2.ii option force application developers to host bootstrap nodes for their applications?
I am leaning towards having one big network(separate from IPFS) that would allow any node to access any namespaced application/overlay network by finding nodes that are advertising themselves as a part of that namespace. That would require us to have only one static set of bootstrap peers for initial bootstrapping where later nodes could bootstrap themselves through a saved dynamic list of more useful peers.
tzdybal commentedon Mar 8, 2021
There is no difference (except the name).
Yes,
2.iiis the most simplified version, where each network is absolutely separated, consists of participating nodes only.Both
1and2.irequires the same amount of development - we need to find peers for exact application. Tagging peers to select those interesting, while participating in broader network seems to be very good option.@Wondertan could you elaborate why you're leaning towards network separated from IPFS? Pros I can see: not cluttering IPFS network, smaller number of peers to check/filter.
liamsi commentedon Mar 10, 2021
I think, if we connect to the main IPFS network, peers will be connected to a lot (hundreds of) nodes and most of them will not have the LL plugin installed. Hence it's not really useful to connect to them.
I was somehow assuming that this is sth that libp2p provides without us having implement the filtering etc our self. Is that not the case?
tzdybal commentedon Mar 10, 2021
It's not "automagical" AFAIK, but we can use existing components.
We can go with DHT peer discovery with using
AdvertiseandFindPeers. After finding peers we can connect to them (open a stream) and (as proposed by @Wondertan) tag them in connection manager, to ensure that we stay connected.This should be the most flexible approach (so
1,2.iand2.iiare supported).tzdybal commentedon Mar 10, 2021
I also asked about this on libp2p IRC channel.
Wondertan commentedon Mar 10, 2021
All of these may become an issue at the beginning of the journey for our network. I would suggest starting independently to learn how to manage infrastructure ourselves and not to break things for PL, as they are really mad about content resolution timings(e.g. one node with the wrong DHTMode option can affect the whole network). Only after becoming confident about managing the network ourselves, we can consider merging with a public IPFS.
BTW, in the ideal world, it would be better for all networks to speak one DHT protocol(maybe even incentivized) to help everyone searching for needed content. However, we are quite far from that ideal future, though we might get to at some point.
liamsi commentedon Mar 11, 2021
Can you share some pointers about what is missing on the DHT?
What are the NAT issues sepcifically?
In which way would the DHT protocol be incentivised?
Wondertan commentedon Mar 11, 2021
It works but there are multiple things missing in the protocol to meet the paper, like this, and there are maaany other practical(not in the paper) things left that are listed as epic issues in the repo. PL was not doing a lot of work on DHT last year as Filecoin does not rely on it that much, as it registers miners addresses onchain for discovery
Running our nodes behind the NAT and misconfiguring them can degrade the public network. There is an automatic solution, but it still fails to detect proper configuration in all cases. While we can ensure that all our nodes are properly configured, we cannot ensure the same for all the test/incetivised network participants, that is why I am suggesting to be independent at the beginning and merge after being confident that our network works well without any other help from 3rd party. Also, through that time NAT workaround solution should mature as well for us not to think about this issue at all.
IDK exactly :) That last paragraph about the ideal world is not really connected to our issue, that was just thoughts that I missed to format properly.
tzdybal commentedon Mar 11, 2021
Summary of our discussion:
2.1:Technical notes:
Server(if node is not reachable (for example behind NAT), it degrades network performance, as the node is advertised, and "failing" connection may take a lot of time).Add peer discovery (#17)