ANts variable connections

[CactusChris]

This could be my imagination but...
I've noticed that people leave and rejoin ANts in order to 'kick start' it. There is a perception that it grinds to a halt eventually. I think that the 'quality' of connected peers makes all the difference. 'Good' peers and the data comes in - other times nothing moves. I have tried removing the slowest peers and just connecting to fas nodes - with little improvement. Gwren implemented a peer change process that discards those peers that are not proxing data that is meant for my node - and this helped - but this weekend (when we had 47 on IRC for a while) the incoming rate has fallen (for me) to below 5% of throughput. This peer discard process works better in small groups rather than larger groups purely because of statistics (I think). A random choice of the (remaining) nodes in a small groups is more likely to connect to a node that is nearer the source data than a random choice in a much larger group. This is a problem that is going to get worse as the network scales up.
A partial answer is to have more people with more connected nodes and increase the connectivity - this carries the danger of even more proxying rather than receiving data, but has a bigger chance of getting a connection nearer to the source data.
Any other ideas anyone?

[CactusChris]

I've just done a bit of simple modelling on a spreadsheet and the connection profile for a set number of nodes shows an interesting pattern, the chances of being within a node or two that is a supplier of data falls off rapidly as the network goes from 20 -> 70 users and then more or less settles - with the biggest change around the numbers we are at now.
Furthermore the most disruptive thing anyone can do is to disconnect and reconnect. The stepwise discarding of 'proxying only' peers is great but the timescales are too long in comparison to the (my percpetion of) leave/join cycles. No sooner does one start to find a better route than someone in the route ducks out and the node that was providing data no longer can. In a chain of (average) 4 nodes the join/leave cycle is four times more disruptive than the lose/find cycle of peer cycling. If I remember correctly the lose peers cycle is about one every 15 mins (??), this makes the cycle relatively slow compared to the (leave/join process x4).
A faster test route process that is only querying for speed of a route could help - as an additional process this could be like the firewall probe and run in parrallel - how I do not know - I'll leave that to others to figure out.
I'll try to build a proper model and see where it goes...if I get time.

[ANtsP2P]

This could be my imagination but...
I've noticed that people leave and rejoin ANts in order to 'kick start' it. There is a perception that it grinds to a halt eventually. I think that the 'quality' of connected peers makes all the difference. 'Good' peers and the data comes in - other times nothing moves. I have tried removing the slowest peers and just connecting to fas nodes - with little improvement. Gwren implemented a peer change process that discards those peers that are not proxing data that is meant for my node - and this helped - but this weekend (when we had 47 on IRC for a while) the incoming rate has fallen (for me) to below 5% of throughput. This peer discard process works better in small groups rather than larger groups purely because of statistics (I think). A random choice of the (remaining) nodes in a small groups is more likely to connect to a node that is nearer the source data than a random choice in a much larger group. This is a problem that is going to get worse as the network scales up.
A partial answer is to have more people with more connected nodes and increase the connectivity - this carries the danger of even more proxying rather than receiving data, but has a bigger chance of getting a connection nearer to the source data.
Any other ideas anyone?

Good points Chris,

Gwren has solution to this but won't be implementing fully (I think) until after 20th Jan 2005 as he is studying for exam.

See http://www.dutchp2pforum.com/portal/viewtopic.php?p=315#315 about stuck downloads

and http://www.zeropaid.com/bbs/showthread.php?t=25411 about IP messaging which in effect moves peers that are file sharing closer together :wings

[CactusChris]

Thanks ANtsP2P for the reply - I looked at the stuck downloads description...

Ezzy said
"Routing Round Blockage Spreads Blockage
However, whenever there is a blockage caused by excess demand or lack of bandwidth, unlike the Internet, it automatically routes around it. So there is a knock on effect and the blockage spreads. Therefore after the network has been running for about 12 hours with about 30 users it reaches an equilibrium state. This is where virtually all network capacity is used to proxy data. The proxy chains are long to avoid bottlenecks. A high proportion of the data is re-sent data. Very little of the proxy data reaches its destination. Most nodes use near a 100% of their bandwidth to proxy data so very little data is download or uploaded. This is what I call a network deadlock. "

This is interesting - but I'm not convinced. The discarding of proxying only peers should solve this problem - and it did seem to at first. I believe that the addition of users is having a much greater effect on the network. As the users grow the proxy chain grows and the networked nodes grow further apart on average. This is why so many people have been suggesting that ANts will not scale. With users leaving/joining the chains are broken and so new routes are sought. Only really active and agressive discarding of peers that are not involved in providing downloads to the local user will solve this as the network grows. This would mean we all could have more peers and discard any that have not handed over any data for the last few mins - the effect would be to break a lot of chains but eventually the system should settle down - if this was the case then the longer one stayed connected the better would be the dls - an incentive to not keep disconnecting.
New nodes that are not yet involved in any transfer are a problem at first - as they will only initially proxy data (while they are looking for some data to download) - but if they are sharing then someone should take a liking to them and they will become involved in a quicker chain.

How did EZZY get the data? Is there some other way to find out what is flowing where that I do not know about or is it mostly supposition?

[ANtsP2P]

Thanks ANtsP2P for the reply - I looked at the stuck downloads description...

Ezzy said
"Routing Round Blockage Spreads Blockage
However, whenever there is a blockage caused by excess demand or lack of bandwidth, unlike the Internet, it automatically routes around it. So there is a knock on effect and the blockage spreads. Therefore after the network has been running for about 12 hours with about 30 users it reaches an equilibrium state. This is where virtually all network capacity is used to proxy data. The proxy chains are long to avoid bottlenecks. A high proportion of the data is re-sent data. Very little of the proxy data reaches its destination. Most nodes use near a 100% of their bandwidth to proxy data so very little data is download or uploaded. This is what I call a network deadlock. "

This is interesting - but I'm not convinced. The discarding of proxying only peers should solve this problem - and it did seem to at first. I believe that the addition of users is having a much greater effect on the network. As the users grow the proxy chain grows and the networked nodes grow further apart on average. This is why so many people have been suggesting that ANts will not scale. With users leaving/joining the chains are broken and so new routes are sought. Only really active and agressive discarding of peers that are not involved in providing downloads to the local user will solve this as the network grows. This would mean we all could have more peers and discard any that have not handed over any data for the last few mins - the effect would be to break a lot of chains but eventually the system should settle down - if this was the case then the longer one stayed connected the better would be the dls - an incentive to not keep disconnecting.
New nodes that are not yet involved in any transfer are a problem at first - as they will only initially proxy data (while they are looking for some data to download) - but if they are sharing then someone should take a liking to them and they will become involved in a quicker chain.

How did EZZY get the data? Is there some other way to find out what is flowing where that I do not know about or is it mostly supposition?

I think he got this data from Gwren. Gwren monitors network traffic when he is trying to debug ANts P2P.

As to solution - there are many solutions. So only time will tell what solution is implemented.

However, IT WILL BE FIXED. :gj

[infringer]

hrmmm interested though I like the security and concept of antsp2p I kinda figured this was the downfall of the security the pooposed incentive for connection is not a bad idea at all as proxing data can in the end turn out to be the letter in the mail from the ISP for overstepping the line of bandwidth usage on your node... This is why currently I would not use it though I think the concept is quite a good one more work and idea needs to be put into the infastructure of the network antsp2p really has no where to go but up once they figure out more ways to slain the bottlenecks and amount of proxing used by one user too bad there wasnt a server or two willing to foot the bill just to proxy the data securely that would be kinda nice but currently though I am very supportive of antsp2p it has some issues to be ironed out.

-infringer-

[tsafa1]

Are you taking netsplits into account. i don't understand it fully, but i have been told by other people more familiar with IRC that it is typical in IRC to see a 10 people drop off at once and then rejoin a few minuts later. Could that be what you see?

regarding stuck downloads, Gwren has said that he is aware of the problem and will try to address it as soon as he clears up some personal stuff. He is currently preparing to take some test.

As far as will restarting, fix the stuck download problem. hmmm... not clear to say. I think i get better results by searching and resumming. You should avoid restarting be cause not only does that disrupt people uploading files from you, but it also disrupts files being proxied through you. If you restart your node ants has to do a lot of rerouting for everyone. That increases the overhead needlessly.

[tsafa1]

... proxing data can in the end turn out to be the letter in the mail from the ISP for overstepping the line of bandwidth usage on your node...
-infringer-

I have heard that some people have limits that is very unfortunate. I hope other options become available in their area soon. I do not have bandwith limits on my DSL line and my provider was kind enough to double the capacity of my connection recently (at no charge !!!) Given that i pay for broadband i feel that i must get my values worth, so i run ants 24/7. Before ants i use to run what ever i felt was the cutting edge program of the time. I have to give intel a lot of credit. I have been running the same motherboard for about 6 years now 24/7 non stop. it won't quit. I have upgraded a cpu, mem. same c drive, just added additional drives.

[CactusChris]

Tsafa and others - some questions :
how do you tell if people are leaving ANts and restarting - or if they are just dipping out of IRC?

Another question... what is the problem with the long delay in 'Getting Hash Map' - it seems to take ages - how big is the hash map and how much computing does it take? say for a 1Mb file.

If I run the Java logger will that tell me if peers disconnected or if I disconnected from them to find a better routing.

How does Gwren get stats in an anonymous network ?

My stats now show under 5% dls vs throughput - it has been as high as 25%. I assumed it was due to 'good' peers - is there a way to tell?

[tsafa1]

sorry no stats realy posible. best you can do is try to peice something together from the traffic passing though your node. I use an external bandwithmeter to see total bandwith usage. I compare that to the internal stats in ants that show you total downloads/uploads and own uploads/downloads. I comparethat to the uptime and compare own transfers to own transfers vs my bandwith capability to see how ants is doing.

Gwren, pust out a minor update today, 8.4 to address stuck transfers. That may fix "getthashmap" delay.

Those simple stats is mentioned showed me that 8.2 (5 hours prior to 8.4) was not doing too well. 8.4 looks good so far but only been running it 2 hours so far. too soon to tell.

regarding how to tell if someone quits or just net split... not sure. IRC is seperate from ants.

[ANtsP2P]

Tsafa and others - some questions :
how do you tell if people are leaving ANts and restarting - or if they are just dipping out of IRC? I read in http://www.mirc.com/ircintro.html Dumping causes network "burps", causing connections to go down because servers cannot handle the large amount of traffic. So maybe that is part of it the irc channel cannot cope with the large number of users so is disconnecting and reconnecting so to those still connected it looks like users are quitting and rejoining.
Also this
(41) What is a netsplit? Why does everybody keep signing on and off?
What does it mean when I see: ***NickName has quit IRC (*.bu.edu eff.org)?

Netsplits are (unfortunately) a routine part of IRC life. The above message means that NickName, who was on a channel with you, was on a different server from you and this server split off from the part of the net you are on.


A -------------- B ---- C
| |
D E

Lets assume a small IRC network where A, B, C, D and E are servers. Let's say that you are on server D, and server A splits from server B. In this case, you will see all users on the servers B, C and E, "sign off". On large IRC networks and crowded channels you will see a huge amount of people 'quiting IRC'. When A and B rejoin, you will see users from B, C and E "rejoin" the channel you were on.
Note that netsplits are all from the point of view of the user. After a netsplit rejoins people might ask where you went -- because from their perspective, *you* split off. The only thing you can do during a netsplit is wait for the net to merge itself. Changing your server during a netsplit is a Bad Idea, because you are likely to have your nickname collided. A "split" often occurs due to faults in the underlying *physical* network. It can also occur due to other reasons, such as if the machine on which the IRC server runs crashes, or if it is too overloaded to handle connections as happens on bigger nets, or if an IRC operator willfully disrupts the connection between two servers to achieve better routing (server - server connections).



Another question... what is the problem with the long delay in 'Getting Hash Map' - it seems to take ages - how big is the hash map and how much computing does it take? say for a 1Mb file.
ANts uses a tree hash system with two layers: a root hash and many sub hashes over fixed size chunks (320k). The root hash is computed over these sub hashes.
The system is a special purpose tree hash adapted to fit ANts net needs... it is not an implementation of any standard tree hash system (tigertree or whatelse). It also has an ed2k hash, chunk is 9.28MB in size,MD4 hash.
So that is alot of hashes. So a 1MB file would have 6 hashes (5 ants root+chunks 1 ed2k chunk). The time I don't know but 8.4 appears to download hash maps much faster than 8.2.



If I run the Java logger will that tell me if peers disconnected or if I disconnected from them to find a better routing.When i last looked the ants log file gave this information. It said when it disconnected due to lack of activity.



How does Gwren get stats in an anonymous network ?Don't know but you can as tsafa points out monitor your traffic with net limiter and ants connection page.



My stats now show under 5% dls vs throughput - it has been as high as 25%. I assumed it was due to 'good' peers - is there a way to tell?I think the own to total traffic percentage reflects the average length of proxy chains (number of hops); re-sent file transfers due to network blockage and number of slow connections compared to fast connections. The slow connections tend to leech proxy bandwidth from the fast connections.

[tsafa1]

Thanks for that very informative post. You cleared up some stuff about how IRC works.