Download speeds will not get much higher, here is why (paper):

[ten9]

I've written a paper about bandwidth issues in P2P networks. Some findings are interesting and easily overlooked. I will present some of them here, and will subsequently discuss them in relation to the various P2P networks.

I'm interested in your opinions on the issue. I'm not here to start a flame war over which network or P2P-client is better. They each have their pros and cons. It depends very much on what you call 'better'.

Better could for instance mean any of the following things:

- Better average download speed

- Better availability of rare files

- Better able to guarantee that every user is treated equally


I'm going to claim that for a program like Shareaza (and the networks it is on), download speeds for an individual user are not going to rise significantly. And specifically the average download speeds are not going to rise above the average upload speeds of all clients.

Sure, average download speeds can get a little better by reducing network overhead or introducing a credit system, but this will not give any large gains.

In this discussion I'm going to assume people want to download large files (movies for instance). For small files things are a little different.

Keep in mind that there is a difference between average download speed, and standard deviation of download speeds. A high standard deviation of download speeds can cause some individual downloaders to have high speeds, and others to have low speeds.


On a P2P network one client cannot, ON AVERAGE, get more download bandwidth for one file than the average upload bandwidth from all supplying other clients. This will be the case because for most clients that share large files, all of their upload bandwidth will be used to supply files to others. People also usually are downloading while they are uploading. I think it is safe to say that on average people at least download as much as they upload. If everybody is on average downloading at least as much as they upload, you will never get on average a larger download speed for a file than the average upload speed.

But, one might say, those are averages, what does that mean for a particular individual downloader?

That is a good question and the answer is that it varies from network to network. How likely is it that you (as a downloading client) will be on or near this average? Or in other words what is the standard deviation of download speeds from that average? It turns out that the more 'fair' a network is, the higher the chances are to reach a download speed that is equal to the average upload speed of all other clients. Higher download speeds (or better: a higher Standard deviation in download speeds) will be possible only if some clients have advantages compared to others (for instance cheats).

If leeching is not punished (through a credit system without cheats for instance), the average will drop.


How do the various networks fit into this?

Kazaa/Fasttrack:

Well Kazaa has a credit system, but it is open for cheats. Kazaa Lite can give you a godlike status on the network, which allows you to jump queues and an 'auto search more'-button to exploit it to the full. (The auto-search more- button is not available to all)

This means some can get good download speeds (even above the average upload speeds of all others), at the expense of others.


WinMX:

WinMX is working on a 4.0 version of the software. The main goal is to reduce the queuing problem the network has. Well, I'm sure it is possible to get the average download speeds a little higher, by reducing some of the overhead. But generally, I'm afraid, the shorter queues will come at the expense of lower download speeds.

Bittorrent:

Bittorrent is a good attempt to maximize the average download speed to the full. It is very efficient.
If you want to read more about the workings of the bittorrent network and scaling issues, read this paper:

http://bitconjurer.org/BitTorrent/bittorrentecon.pdf

The only way to get higher download speeds than the average upload speed from all clients here is, if people leave the torrent open, when it has finished downloading, provided that they are not uploading other things at the same time. By using the bittorrent implementation present in Shareaza, this rule is usually broken, because people are usually also connected to the Gnutella 2 network (and sometimes eDonkey too). So if everybody would use Bittorrent, instead of Shareaza, average download speeds would be somewhat better. I have to add that the torrent implementation of Shareaza takes away most of the upload bandwidth from other networks it is connected to, so possibly the difference will not be that large.


Gnutella 2:


Gnutella 2 is a network without any cheats (as far as I know), without a credit system and with the ability to search the whole network. The average download speed will be lower than it could be, because of the lack of any credit system to punish leechers. Standard deviation of download speeds will be relatively low, because there aren't any cheats. It can be considered a 'fair' and well-balanced network, contrary to Fasttrack/Kazaa.

The Gnutella2 network is growing rapidly but the download speed will basically remain the same. There will always roughly be as much sources for files as you will get downloaders. Shareaza's ability to swarm several networks is good for finding rare files, but will not help you in getting large files any faster.


Conclusion:

As long as the upload speeds of clients are way below the download speeds on average, P2P will not allow us to benefit from the download speeds we get from our Internet Service Providers.

As long as they will give us download-upload ratios of 4:1 (like in 1024 Kb - 256Kb) we will generally not get average download speeds higher than 256Kb.

For that to change, ISP's need to give more balanced download-upload ratios. But surely ISP's do not want that, because now they can benefit from selling contracts for download speeds that are generally not reached.


Downloading small files will generally be possible with relatively high download speeds. This is because there are always people that have small files (like mp3's) available, and do not have large files available, so that their upload bandwith is not used for other things all the time. Having small- and medium filesize queues helps to keep smaller files available when clients supply both large and small files. This is why I suggest that Shareaza should generate these queues by default when shareaza is installed.


Regards, Ten

ten9@yahoo.com

[Malicious Intent]

I've thought before that the natural selfishness of people to only worry about their download speed when chosing an ISP would have an adverse affect. What you need to remember is that people are not always downloading, but upload is always in the background. On most p2p apps this is counteracted by leechers and those that go on grab what they want and then leave.
Without figures on how people set up their computers and how they chose their ISP, it will be imposible to say that downloads will never go above 256kb. This rule clearly doesn't work for BT. I limit my upload to 16k, but still get a download of over 100k=800kbits. I've been as low as 5kb and still got those speeds. I always continue to seed until my up:down=150%. Then there are all those that log off straight away, but speeds continue to be well above 800kbits, not 256.
Good theory, but is it reflected in practice?

[Criminal_Sniper]

i think the networks need slave servers hooked in like clients with big bandwidths and lots of verifieds

and why the fuck do i have 4300 sources on edonkey2000(BC)
and hey wait it might do the last 0.5% percent in this hour
edonkey just simply sucks for speed
i though it was user count before

[Afn]

Over time the bandwith issue will be resolved. on broadband what took an hour on 56 k takes minutes, and this is a trend that will continue.

The vested interests, the media trusts do not want the isp's to offer 700 meg download speed in let's say 5 minutes. It would be great to beable to download a video faster than you can watch it.

It will be a matter of time and we will have faster speeds. Right now, the media trust is making isp's pay them or no play.

In the end faster downloads than you can watch, and more selection than the few lame offerings from your local riaa or mpaa trust.

[squirm]

We may have faster speeds but we'll want more. For example; higher bitrate files=larger files.

[MonkeyMadness]

When I first read the subject I thought this was going to be a "why DSL / cable is as fast as it can go" sort of topic, and I was reminded of when 1200 baud was the supposed absolute limit for phone lines... then someone invented trellis modulation and whoooooooo we jumped to 2400, 9600, eventually 14.4K, 28.8K, and 33.6 was the "absolute limit"... then someone decided hey, we want faster downloads, not so much uploads... what if we exploited the upstream charateristic to make it faster in one direction? suddenly another limit is broken and we have 56K (actually legally limited to 53.somethingK) download speeds... likewise, DSL and cable tech will improve, first without and later with infrastructure changes... one day we'll all have a fiber drop. I also remember someone once claiming that there was no way, considering the resolution of a television and such, that video game graphics could be better than what was offered on the Atari 2600. Obviously they were wrong too :) But none of that is precisely relevant to what we seem to be discussing, lol.

So.... to get vaguely on topic with my mini-rant, Malicious Intent has a great phrase in his reply: "the natural selfishness of people". The automatic assumption by most people is that this is bad. I disagree. If it is "natural" then we'd be far better off working with it, then against it. So much effort is spent trying to make things "fair", with anti-leech code, and software that doesn't allow sharing to be turned off, and centralized networks that try to maintain control by disallowing other software... *sighs* In the end its wasted effort, those who wish to leech, or write their own version of such-and-such client, will succeed. Better off to work with those people then make enemies, today's leecher could be tomorrow's host of 250 gig of rare files. I think those who write P2P software, and the "inner circle" of those who advocate P2P, are often of the opinion that everyone should think like them, and then we'd all have a great P2P service. On the contrary, you can't change the way the vast majority of people think (with rare exceptions). You need to cater to the needs of the vast majority, who want files, now, with no hassle or risk, or you will not have the vast numbers of converts (it is a bit like religion, no?) needed to have a successful (think Napster of Kazaa at their peak, being household names) service. Think about it for a bit... why did people flock to Napster? "free music, just click, its easy". Why did people flock to Kazaa? "its like napster only better, images and software and movies too". Very few people signed on to be altruists and share as much as possible.

[Killawat]

You can't really put an average speed on filesharing networks. Because its based on 2 people, the reciver and the sender. Well eddy, you said you got broadband yesterday, so why is this transfer still maxing out at 3 KBps. Because YOUR connection dosn't allow the maximum that my connection can dish out. So lets say 5 modemers all want 1 file from 1 broadbander. If the broadbander has a 30 KB upload. Every1 will be happy and get 5 kbs. But if the broadbander only has 12 KB's. They will get 2-3 kb's. Bittorent is another example. They way they share your upload through a wide spectrum giving parts to people who need them. It would be impossible to figure a system that would say "well hey this guy is on more and on a faster connection, lets give him these files and it'll spread quickly". which is why kazaas participation scheme got screwed in the ass. But yea, bittorent does this best because lets take 5 broadbanders and 1 broadbander with the file. All 5 of them start off getting the file (the first peice). Lets say 2 of them get off but the other 3 stay on. Then they get 2 peices. Then the host gets off. But the other 2 that got off before get on and the original 3 that stayed on are still there. If those 3 have lets say a 20 kb upload and they MAX IT OUT then those other 2 would probably get 30 kbs ::depending on who is faster:: and the file would continue to spread like that. Always getting a slow part from the host ::with a 24 kb upload:: and then the other 3 giving it up.

[John W. Lindh]

Thank you, ten9 for this exceptionally beautiful example for how flawed a paper can be if you don't do any research at all. I'm sure it must've been hard to avoid anything that could be called even remotely solid facts and figures.
And the way you put all those half-truths that everyone who successfully made it through middle school could have figured out on his own in shape of a pseudo-scientific paper. I'm sorry, I have to repeat myself. That was just beautiful. Thank you.

[Killawat]

Thank you, ten9 for this exceptionally beautiful example for how flawed a paper can be if you don't do any research at all. I'm sure it must've been hard to avoid anything that could be called even remotely solid facts and figures.
And the way you put all those half-truths that everyone who successfully made it through middle school could have figured out on his own in shape of a pseudo-scientific paper. I'm sorry, I have to repeat myself. That was just beautiful. Thank you.
well that was mean
but i gotta admit.
PWNED!

[ten9]

John, could you at least point out what these half-truths are?
It is actually based on a simple mathematical model, that will be presented in an article later.
Anyway, things may be obvious to you, but surely not to a lot of people. Even some developers have shown (by means of their design choices) that they do not get this underlying very simple rule in P2P networks.
But, be constructive and show me exactly what is flawed.


Thank you, ten9 for this exceptionally beautiful example for how flawed a paper can be if you don't do any research at all. I'm sure it must've been hard to avoid anything that could be called even remotely solid facts and figures.
And the way you put all those half-truths that everyone who successfully made it through middle school could have figured out on his own in shape of a pseudo-scientific paper. I'm sorry, I have to repeat myself. That was just beautiful. Thank you.

[John W. Lindh]

Did that sound so mean? I'm sorry, it certainly wasn't meant that way, I was just getting a little bit excited. I need my china girl back....to say 'sh-shhhhh'.

But honestly, where does someone come up with the idea that reducing queue lengths would reduce download speed. It just means that instead of serving files first-come-first-serve you serve the file to those who are lucky enough to get a slot. If you view the big picture the overall uploaded data remains the same, - although the overhead may be reduced slightly by making queues smaller.

In addition, credit systems don't per se increase the overall uploaded data either although they may encourage other users to upload a little more. However this effect is purely psychological since, even if there is some gain by uploading more, you are still being able to download easily without uploading. So instead of uploading more you could just spend that bit of upstream bandwidth on creating connections, asking a lot more sources for files & so on. In large networks you don't often meet the same node twice.

In smaller networks (like G2) a credit system could work to some extent but I doubt that you could measure the effect.

And while ten9 was ranting about how cool it is to swarm from multiple p2p networks at once, he might have given just a little bit of thought to the fact that connecting to more networks also means more overhead, ultimately reducing the amount of uploaded data. In the end everybody can download less except for those few who take up the resources of multiple networks.

[thongsai]

u just stated the obvious. we can download fast but cant upload fast.

[John W. Lindh]

John, could you at least point out what these half-truths are?
Yes, I could - and I did.

It is actually based on a simple mathematical model, that will be presented in an article later.
I'm absolutely thrilled. When will we get to see ten9's Law of Conservation of Matter, Energy and Bandwidth?

Anyway, things may be obvious to you, but surely not to a lot of people. Even some developers have shown (by means of their design choices) that they do not get this underlying very simple rule in P2P networks.
Yeah, especially *cough* developers *cough* of a certain *cough* client that just happens to be connecting to multiple networks.

But, be constructive and show me exactly what is flawed.
I'll try... but it's ... hard to be constructive ... when your nuts a swollen ... and your china ... girl ... isn't ... there. Gotta call up that mail-order bride company again, - maybe they'll give me a refund or something.

[ten9]

A credit system will cause more leechers to actually donate their upload bandwith to the community. Especially if they notice that their downloads aren't going that well. It won't stop them from downloading though. So obviously they have a very negative effect on the community. Of course the effect is psychological, so what? The effect is there.

I never said it was 'cool to swarm all those networks'. I only said it is useful to find rare files.

=But honestly, where does someone come up with the idea that reducing queue lengths would reduce download speed. It just means that instead of serving files first-come-first-serve you serve the file to those who are lucky enough to get a slot. If you view the big picture the overall uploaded data remains the same, - although the overhead may be reduced slightly by making queues smaller.

In addition, credit systems don't per se increase the overall uploaded data either although they may encourage other users to upload a little more. However this effect is purely psychological since, even if there is some gain by uploading more, you are still being able to download easily without uploading. So instead of uploading more you could just spend that bit of upstream bandwidth on creating connections, asking a lot more sources for files & so on. In large networks you don't often meet the same node twice.

In smaller networks (like G2) a credit system could work to some extent but I doubt that you could measure the effect.

And while ten9 was ranting about how cool it is to swarm from multiple p2p networks at once, he might have given just a little bit of thought to the fact that connecting to more networks also means more overhead, ultimately reducing the amount of uploaded data. In the end everybody can download less except for those few who take up the resources of multiple networks.

[shawners]

Best ANTI LEECh program their was, AUDIOGALAXY.. TO DOWNLOAD 2, you had to allow an upload of 2.. Just plain simple, no closing connections.. BUT The thing about it is, when your ready to download what you queued, you had to turn the satelite on, so that could cause you to be leeching, and you should always have to share a directory, but people will manipulate it.... QUEUES is what kills networks when you can choose to download from DSL/cable users.. And people lies about it so no one downloads from them.. EXAMPLE.. winmx. IF i created a P2P agent. What it would have is closing all other clients off.. Have to have 10 gigs of media, verfied and cant turn it off.. No settings to turn bandwidth limiton or off/ Yet other 3rd party software can allow it or make it where you can limit the upload.. so all in all.. NO matter what you make or do, people will take advantage of your system, use hacks or other sources.. So what sets us apart, brings us together again. We have to trust each other and hope that everyones doing their part of sharing.

[John W. Lindh]

A credit system will cause more leechers to actually donate their upload bandwith to the community. Especially if they notice that their downloads aren't going that well. It won't stop them from downloading though. So obviously they have a very negative effect on the community. Of course the effect is psychological, so what? The effect is there.
You are talking to a long-time mldonkey user here. - Believe me, my downloads can go VERY well - without uploading, - even when accessing a certain network that has some kind of credit system. Like I said, instead of uploading, I can just start HAMMERING the network for download slots using 1,000 parallel connections. For every user you can convince to increase his upstream, there could be one or two using one of those leecher-tools.

I never said it was 'cool to swarm all those networks'. I only said it is useful to find rare files.
I'm sure if you can't find certain files on the edonkey network, the G2 network will be very useful. But you gotta ask yourself, how often do you need to find those unbelievably rare files, - and how long do you use up resources of multiple networks.

BTW. do you think it's a bad sign if a woman that is visiting her parents for a couple of weeks suddenly stops answering your emails and won't give you the number of her new mobile phone???

[DudeAsInCool]

As I understand it, the Internet is an efficient network for large downloads, because it wasn't designed for them. Would an optical network be more efficient?

[John W. Lindh]

As I understand it, the Internet is an efficient network for large downloads, because it wasn't designed for them. Would an optical network be more efficient?
Yes, but only if you make the cables big enough to fit whole DVDs.

[ten9]

Ok, you are just being an ass. My post you are replying to was there BEFORE you came up with a bit of an explanation. So this post was absolutely not necessary, was it?
If you have no intention to be constructive , go find that China girl.



Yes, I could - and I did.


I'm absolutely thrilled. When will we get to see ten9's Law of Conservation of Matter, Energy and Bandwidth?


Yeah, especially *cough* developers *cough* of a certain *cough* client that just happens to be connecting to multiple networks.


I'll try... but it's ... hard to be constructive ... when your nuts a swollen ... and your china ... girl ... isn't ... there. Gotta call up that mail-order bride company again, - maybe they'll give me a refund or something.

[REDO]

damn, its been a while since i replied to a topic on here, but I think DL speeds will increase exponentially over the next decade. I think we will see the price of t1 lines, and fiber-optic lines into consumers homes drop in installation price, and increase in use. Thus, increasing bandwidth overall.

[Killawat]

dayum Ten9, chill out. We just wanted to point out the different flaws so u don't get critizied in the class or something. Please memorize my post about the way bittorent works for instance :D

[muffenme]

:fire

I do agree that there always going to be leecher, new tech., and people that would love to share what they have.

I like to see the end of virus, spyware, Adware, leecher, RIAA and MPAA.

:hole

[Malicious Intent]

dayum Ten9, chill out. We just wanted to point out the different flaws so u don't get critizied in the class or something. Please memorize my post about the way bittorent works for instance :D
Which I don't THINK is completely right. I found this by an unknown author (to me) in an article called "the robustness of the bittorrent protocol"

"Bittorrent also incorporates “optimistic unchoking” for discovering better peers, i.e. peers who will upload to it much faster. Bittorrent peers upload only to a subset of the peers they are connected to, called the preferred peers. In optimistic unchoking, a peer picks another peer not already among its preferred peers and uploads to it, in the hope that the peer will reciprocate. If this remote peer uploads at a rate faster than any of the preferred peers, then a new preferred peer has been discovered and it displaces the slowest preferred peer. This ensures that a peer is always progressing towards better bandwidth utilization."

Which surely means that faster peers slowly group together as they make eachothers preferred lists.