Rendered at 23:49:53 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
swax 7 hours ago [-]
I did a client called Gnucleus back in the day, the original website is still up at https://gnucleus.org/ I also designed GWebCache with a friend in college (also the IRC caching before that was me, I'm sorry). I think in the end the cost/benefit of the Gnutella architecture wasn't there. Especially in privacy - too much of a liability exposing your machine to serve files and route traffic. Also the efficiency of search over the Gnutella architecture - your query never hit all the peers.
BitTorrent was designed with the good parts like decentralized file transfer, and ditched the decentralized search - simplified with a centralized website/tracker model that can also be members only. That helped people stay more under the radar, as well as allow people to jump on/off to get what they want.
Ultimately the better balance which is why BitTorrent is still going strong today, but there is some nostalgia for the craziness of a single global network. Where people can freely share all their stuff, and downloading/opening files was like rolling the dice.
Mc91 5 hours ago [-]
I remember Gnucleus and followed the release of Gnutella and used to read the old Gnutella Developer Forum on Yahoo where the major Gnutella clones hashed out the next steps for the network.
I agree - competition from BitTorrent lowered Gnutella's popularity.
Another thing about BitTorrent - I go to the Pirate Bay right now and look at top 100 for music, and "Pink Floyd - Discography 1967-2014" is one of the top torrents - 2.86 gigs, 131 seen seeds, 10 seen partials (seeds means they have 100% of the blob). You can download all their albums if you want, or one album, or one song on one album. Also most bittorrent clients have anti-leech constraints, so someone might start out accepting and sharing any part of that blob, making the one song they want a high priority request for them - they get the song they want quicker but they're also contributing to the torrent while they're online. So this sort of thing has all kinds of benefits for the network.
Whereas Gnutella tends to be look for and get one song at a time.
rickcarlino 7 hours ago [-]
Thanks for dropping a comment! Collecting Gnutella history is a hobby of mine if the article did not make that apparent. I’ve seen a lot of your past work related to Gnucleus while researching the protocol and digging through old documents in archive.org. It always surprised me that the homepage was still online after so many years.
Are you still active in open source / decentralized tech these days?
swax 6 hours ago [-]
Yea, I'm working on a open source distributed, self-hosted agent runner https://naisys.org/
I often wonder why we don't go back to systems like WASTE now.
We all finally have the bandwidth.
qubex 3 hours ago [-]
As the eldest of millennials, I regularly find myself going through “ah, they’ve reinvented a situation best served by Gnutellla” events. The most most recent was an ad hoc Resilio Sync library of epub files, and I did the “yeah that works… but have you ever heard of Gnutella?” routine I have become so accustomed to: “gather around the camp fire, young zoomers, for grandpa has a story to tell you”.
abcd_f 5 hours ago [-]
The main issue with Gnutella was - IIRC - that it didn't scale. At least the initial version, not sure if there was a revised one.
Basically, if you open the log window and look at the peer messages, then beyond certain network size all you'd see was a flood of relayed search queries with duplicates that ground all other activity to the halt. And the whole thing just became unusable.
PS. Also, Gnutella was released almost to the day when some clause of the AOL's purchase contract of Nullsoft (stock option vesting?) has expired so the devs were ultimately free to do whatever the f they wanted. So released the file sharing app. That was a nice touch.
gbildson 5 hours ago [-]
We made it scale with last hop query routing. Basically we realized that 90% of the query traffic occurred on the last hop (ultrapeer to ultrapeer). So we took the keyword bloom filter we were already using on leaf node connections, aggregated those and propagated that to adjoining ultrapeers. Then we sampled response rates of queries from originating ultrapeers with increasing TTLs to get a sufficient but not overwhelming results. We also increased the ultrapeer to ultrapeer connections so that we could sample more efficiently while dropping the TTL.
That was LimeWire connections and other clients like BearShare and gtk-gnutella and possibly Gnucleus that obeyed the new protocol requirements. Others were kept in a sequestered network and deprioritized.
Sophira 3 hours ago [-]
I'd say the biggest reason that Gnutella (and other services like it) is no longer in much use is because for a long, long time now the easiest way to download music for free, without ads, and with virtually zero chance of getting caught, has been via YouTube and downloader clients. For most practical uses, it's good enough.
You wouldn't want to share the resulting downloads (not only is the audio quality slightly degraded, but I imagine it's highly likely there would be audio watermarks), but when everybody can download straight from YouTube anyway with a minimum of hassle, why would you need to share anything other than a video URL?
Of course, a big part of why this is so simple is because of the massive amount of work that the downloader client devs put into working around YouTube's attempts to stop this. I imagine it can be a difficult job.
If YouTube ever win the battle against the downloader clients, I imagine the landscape will change again. Maybe Gnutella will make a comeback.
d3Xt3r 3 hours ago [-]
It's got nothing to do with YouTube - people just don't download music any more, period. At least not like they used to, back in the late 90s and 00s. Instead, most people these days stream music from the myriad of streaming services, and yes even YouTube - it's a convenience thing. Of course, there are exceptions like audiophiles, indie/niche music, or general offline usage (flights, roaming etc); but the vast majority just stream these days because it's so much more convenient..
kazinator 3 hours ago [-]
I have not downloaded in a long time because ... well ... I downloaded all the good music years ago and they mostly stopped making it. :)
Findecanor 2 hours ago [-]
Many Youtube-to-mp3 sites exist, though.
lnrd 3 hours ago [-]
Spotify killed the concept of downloading music, either legally or not.
kazinator 3 hours ago [-]
After YT downloading and what not, there is still Bittorrent.
kderbe 8 hours ago [-]
I would read a follow-up about LimeWire's dynamic query routing. I enjoyed the writing style very much and now I'm reading Rick's other articles on topics that normally wouldn't interest me. Thanks Rick!
rickcarlino 7 hours ago [-]
I’m glad you liked it! I hope to publish the follow up soon
dlenski 7 hours ago [-]
Great article. It's been almost 20 years since I've used Gnutella, but I seem to recall that, as flexible and resilient as it was, it had some pretty major issues with speed of queries and scaling to millions of users. I think Bittorrent become popular in the mid-to-late '00s in large part because its model of redundant copies scaled better than Gnutella.
> The Gnutella project began as an internal demo that leaked to the public after its corporate overlord, AOL, cancelled the project.
I don't think it was actually a leak in the usual sense of the word. There was no unauthorized release; rather, AOL didn't really understand what their new subsidiary was releasing.
As I understand it, Gnutella was written as a new project by Justin Frankel, the author of Winamp (which probably did more to popularize using computers to listen to music than anything else!), during or shortly after the sale of Winamp and Nullsoft to AOL. It was probably a chaotic time, and a massive culture clash between this big behemoth of a late-'90s tech and communications company, and this small startup of Early Internet Nerds.
Nullsoft's new corporate overlords probably didn't understand what they were creating: a new file-sharing/music-piracy program that would be like Napster, but more decentralized and resilient.
Frankel/Nullsoft released Gnutella and it was downloaded by thousands of people immediately. Perhaps friends in the recording industry called AOL execs, or in some other way they finally understood what it was, and AOL shut down the downloads less than a day later and cancelled the planned open-source code release, but due to its decentralization the network kept running, and it was soon reverse engineered.
As far as I know, the source code of the original Windows implementation of Gnutella never leaked.
gbildson 4 hours ago [-]
Shout out to Gene Kan (https://en.wikipedia.org/wiki/Gene_Kan) who was the intermediary at least for the release of the Gnutella spec and reference client from Nullsoft and an early supporter. He died too young.
nntwozz 3 hours ago [-]
On a tangent OiNK's Pink Palace never died either, if you know you know…
The high seas have never been better.
gitowiec 8 hours ago [-]
Thank you for reminding me about this. Next to soulseek I'm going to use it to not obey!
3 hours ago [-]
pndy 7 hours ago [-]
Didn't soulseek had some requirements you had to fulfill before downloading?
specproc 6 hours ago [-]
I've used soulseek pretty recently! There's a nice client called nicotine, IIRC.
There are message bots that spam you with racist abuse if you're sharing nothing (i.e. not even your downloads), but that's about it.
pndy 6 hours ago [-]
Oh so pretty much the "classic" share something rule; that was pain in the ass in DC++ times but quality of whatever you could find there was best
doublerabbit 5 hours ago [-]
DC++ now that's some nostalgia.
Myself used IRC XDCC bots connected to my 33k modem connected to my 486DX which would disconnect every three hours.
eDonkey, WinMX, lovely fashionable protocols.
abcd_f 5 hours ago [-]
Not to forget the eMule.
M95D 5 hours ago [-]
WinMX
mrenzo 6 hours ago [-]
I always find this kind of thing interesting. A protocol can outlive the companies, devices, and culture around it, mostly because nobody has to keep owning it for it to keep making sense.
chuckadams 8 hours ago [-]
Hot take: the simple reason Gnutella declined is that it was replaced by Bittorrent.
gbildson 4 hours ago [-]
Gnutella's original sin was that it combined distributed search with distributed download. In a rational world, that would be smart and good but in a litigious world, that was too sophisticated for the Supreme Court and they ruled it as infringing on copyright through inducement of the user. Gnutella clients, like other P2P clients with search, got sued out of existence.
Bittorrent offloaded the distributed search onto websites which routinely got sued or shutdown. Funnily enough, one of my big improvements to Gnutella in the first year of LimeWire was to drive out the website users because they were overwhelming the network upload capabilities without adding to them. That improved the 90% download failures in 2001 but interesting to wonder what if we had gone another way.
GuB-42 2 hours ago [-]
I don't think it has much to do with legality.
For me the biggest problem is authenticity. On most of the decentralized P2P networks of that time is that you never know what you are getting. Look for some Disney movie, get porn. Very common back in the days. Of course you also had all the malware, scams, etc... More generally, it didn't do much to incentive good behavior (sharing).
Bittorrent, with its private trackers had some accountability, ratios, etc... The protocol itself favors peers that give the most in return.
UncleEntity 6 hours ago [-]
Sure, but the attached chat rooms were pretty handy, I used to like to download bootlegged concerts back in the day, to find new ones you've never heard of.
Plus, always fun to get laughed for mistyping The Almond Brothers Band at 3am...
outside1234 7 hours ago [-]
I don’t think that’s a hot take, BitTorrent learned from Gnutella and made a better protocol. Gnutella is important historically, but it had a lot of downsides as a protocol that BitTorrent improved on.
superkuh 3 hours ago [-]
>Imagine we all had HTTP servers running on our laptops and could give our friends an IP address whenever we needed to transfer a file. In theory, an HTTP server on everyone's machine would be enough for file sharing, right?
Opera browser at one point included a personal webserver built in called Opera Unite. This webserver was accessed by the public through the free web proxy service that Opera also integrated. That way your personal home computer hosted website was behind an Opera IP address. It was all automagic. I really wish it had caught on. Simple static only hosting is a very small attack surface.
Firefox recently started offering VPN services built in to the browser. I think this is the perfect time to try to bring back static personal webservers in the web browser.
bellowsgulch 2 days ago [-]
That’s what Limewire used? It definitely came pre-bootstrapped then.
rickcarlino 2 days ago [-]
Are you asking if lime wire used Gnutella Web Cache for bootstrapping? I’m not sure. GWebCache is one of many possible ways to boot strap, and I have not run lime wire in over a decade. I saw that GTKGnutella moved off of GWebCache sometime ago and uses some sort UDP based tool now. I am fairly certain that Shareaza still uses it because I see those results come up in my Web cache pull from time to time. I have seen a few advertisements from lime wire fork projects as well.
bragr 7 hours ago [-]
If I recall, proprietary clients usually shipped with their own bootstrap server. I think it may have even contributed to the legal cases, but it's been a long time.
itsthecourier 3 days ago [-]
just reading gnutella triggered a really old memory of times when Ares, Limewire and eMule where places to try your luck getting mp3s and software
felooboolooomba 8 hours ago [-]
Back in the day as a teenager. Downloaded mp3 that was labelled with title and artist and .mp3 extension. It wasn't. What it was caused me to wipe my hard drive and reinstall everything. Fkuc that shit. Apart from that, many good stuff was had.
leeoniya 8 hours ago [-]
or was it Windows hiding file extensions by default and you downloaded a .mp3.exe file?
netsharc 8 hours ago [-]
Isn't it just great how a decision made by some genius in Microsoft decades ago caused so much confusion and mess. Even on Windows 11 the default is to hide extensions, because, geez, wouldn't want to confuse people with change after decades of it being like that.
Although, was the hiding something that the Mac introduced?
The idea of the last part of the filename (after the period) determining what program is launched to handle the file is odd anyway...
I wonder if the Windows spyware infrastructure measures what % of people turn off extension hiding..
chuckadams 8 hours ago [-]
The mac started out without using extensions at all, the type was embedded in the metadata. That's still possible now, but it's largely derived from extensions first. I believe Finder shows all extensions by default. It certainly does in details mode.
throwawaysoxjje 8 hours ago [-]
Macs originally didn’t have filename extensions because the file type was stored as metadata in the file system
hnlmorg 6 hours ago [-]
That really is a superior way of doing things too. Or at least it would have been if that metadata were transferred with the file itself in all protocols.
felooboolooomba 6 hours ago [-]
Nope. It had been renamed from .avi. CP.
GaryBluto 4 hours ago [-]
You'd think it wouldn't be that hard for a client to automatically check if a file is what it claims to be after downloading.
culi 8 hours ago [-]
Docuwiki (not to be confused with DokuWiki) is still the most thorough source I've seen for niche documentaries.
And yet it doesn't work without JS (I think it's because cloudflare WAF, but still)
cwnyth 6 hours ago [-]
Once you get past Cloudfare, you can use it without JS.
cwnyth 6 hours ago [-]
BearShare, too, which also included pictures and video. I don't remember ever getting a virus from it, but I also kept extensions visible.
suzzer99 6 hours ago [-]
I still get spam emails on the one-off email address I used to sign up for Bearshare.
rrgok 8 hours ago [-]
Shareaza was the goat. It had 4 or 5 protocols.
rickcarlino 7 hours ago [-]
It is still around! I see network traffic from it on major GWebCache instances.
dnnddidiej 5 hours ago [-]
Is Bittorrent connected to this. Isn't that still going strong?
magic_hamster 5 hours ago [-]
Nice writeup. It goes deep into Gnutella, but it's also worth mentioning the slew of sharing programs back then, which was truly like the wild west. Napster, Emule, DC++, Kazaa, to name a few. On many of these networks it was possible to literally browse other people's sharing folders, find really cool stuff, and maybe make some guesses on the what this user was like.
thiscatis 8 hours ago [-]
Great memories of limewire but unfortunately its creator has gone full MAGA/MAHA, dropped all scientific knowledge he ever had by funding RFK Jr. and is even advocating cancelling child vaccination schedules.
dnnddidiej 5 hours ago [-]
Does that matter in terms of the software. It is GPL. Shows how good GPL is as an idea. Even if he went heavy pro big tech his code can still be used.
I swear I think we're living in the episode of TNG with the brain worms every day now.
pessimizer 5 hours ago [-]
Bizarre that in the extreme partisan mind not committing to supporting current child vaccination schedules somehow overrules
> Gorton is involved in various green lifestyle issues, especially those having to do with transportation. At one point, Gorton was the single largest supporter of Transportation Alternatives, the New York City-based advocacy group for pedestrians, cyclists, and public transit.[10] In 1999 he founded OpenPlans, a non-profit organization that developed GeoServer, a collaborative open source project encouraging green urban planning initiatives.[11] In 2009 Utne Reader named Gorton one of "50 visionaries who are changing your world".
BitTorrent was designed with the good parts like decentralized file transfer, and ditched the decentralized search - simplified with a centralized website/tracker model that can also be members only. That helped people stay more under the radar, as well as allow people to jump on/off to get what they want.
Ultimately the better balance which is why BitTorrent is still going strong today, but there is some nostalgia for the craziness of a single global network. Where people can freely share all their stuff, and downloading/opening files was like rolling the dice.
I agree - competition from BitTorrent lowered Gnutella's popularity.
Another thing about BitTorrent - I go to the Pirate Bay right now and look at top 100 for music, and "Pink Floyd - Discography 1967-2014" is one of the top torrents - 2.86 gigs, 131 seen seeds, 10 seen partials (seeds means they have 100% of the blob). You can download all their albums if you want, or one album, or one song on one album. Also most bittorrent clients have anti-leech constraints, so someone might start out accepting and sharing any part of that blob, making the one song they want a high priority request for them - they get the song they want quicker but they're also contributing to the torrent while they're online. So this sort of thing has all kinds of benefits for the network.
Whereas Gnutella tends to be look for and get one song at a time.
Are you still active in open source / decentralized tech these days?
I also did a private decentralized system that was like WASTE+DHT around 2009 (https://github.com/swax/DeOps)
I often wonder why we don't go back to systems like WASTE now.
We all finally have the bandwidth.
Basically, if you open the log window and look at the peer messages, then beyond certain network size all you'd see was a flood of relayed search queries with duplicates that ground all other activity to the halt. And the whole thing just became unusable.
PS. Also, Gnutella was released almost to the day when some clause of the AOL's purchase contract of Nullsoft (stock option vesting?) has expired so the devs were ultimately free to do whatever the f they wanted. So released the file sharing app. That was a nice touch.
That was LimeWire connections and other clients like BearShare and gtk-gnutella and possibly Gnucleus that obeyed the new protocol requirements. Others were kept in a sequestered network and deprioritized.
You wouldn't want to share the resulting downloads (not only is the audio quality slightly degraded, but I imagine it's highly likely there would be audio watermarks), but when everybody can download straight from YouTube anyway with a minimum of hassle, why would you need to share anything other than a video URL?
Of course, a big part of why this is so simple is because of the massive amount of work that the downloader client devs put into working around YouTube's attempts to stop this. I imagine it can be a difficult job.
If YouTube ever win the battle against the downloader clients, I imagine the landscape will change again. Maybe Gnutella will make a comeback.
> The Gnutella project began as an internal demo that leaked to the public after its corporate overlord, AOL, cancelled the project.
I don't think it was actually a leak in the usual sense of the word. There was no unauthorized release; rather, AOL didn't really understand what their new subsidiary was releasing.
As I understand it, Gnutella was written as a new project by Justin Frankel, the author of Winamp (which probably did more to popularize using computers to listen to music than anything else!), during or shortly after the sale of Winamp and Nullsoft to AOL. It was probably a chaotic time, and a massive culture clash between this big behemoth of a late-'90s tech and communications company, and this small startup of Early Internet Nerds.
Nullsoft's new corporate overlords probably didn't understand what they were creating: a new file-sharing/music-piracy program that would be like Napster, but more decentralized and resilient.
Frankel/Nullsoft released Gnutella and it was downloaded by thousands of people immediately. Perhaps friends in the recording industry called AOL execs, or in some other way they finally understood what it was, and AOL shut down the downloads less than a day later and cancelled the planned open-source code release, but due to its decentralization the network kept running, and it was soon reverse engineered.
As far as I know, the source code of the original Windows implementation of Gnutella never leaked.
The high seas have never been better.
There are message bots that spam you with racist abuse if you're sharing nothing (i.e. not even your downloads), but that's about it.
Myself used IRC XDCC bots connected to my 33k modem connected to my 486DX which would disconnect every three hours.
eDonkey, WinMX, lovely fashionable protocols.
Bittorrent offloaded the distributed search onto websites which routinely got sued or shutdown. Funnily enough, one of my big improvements to Gnutella in the first year of LimeWire was to drive out the website users because they were overwhelming the network upload capabilities without adding to them. That improved the 90% download failures in 2001 but interesting to wonder what if we had gone another way.
For me the biggest problem is authenticity. On most of the decentralized P2P networks of that time is that you never know what you are getting. Look for some Disney movie, get porn. Very common back in the days. Of course you also had all the malware, scams, etc... More generally, it didn't do much to incentive good behavior (sharing).
Bittorrent, with its private trackers had some accountability, ratios, etc... The protocol itself favors peers that give the most in return.
Plus, always fun to get laughed for mistyping The Almond Brothers Band at 3am...
Opera browser at one point included a personal webserver built in called Opera Unite. This webserver was accessed by the public through the free web proxy service that Opera also integrated. That way your personal home computer hosted website was behind an Opera IP address. It was all automagic. I really wish it had caught on. Simple static only hosting is a very small attack surface.
Firefox recently started offering VPN services built in to the browser. I think this is the perfect time to try to bring back static personal webservers in the web browser.
Although, was the hiding something that the Mac introduced?
The idea of the last part of the filename (after the period) determining what program is launched to handle the file is odd anyway...
I wonder if the Windows spyware infrastructure measures what % of people turn off extension hiding..
https://docuwiki.net/index.php?title=Category:Name
It is all ed2k links. Unfortunately modern clients for ed2k are quite lacking
holy smokes you're right!
https://en.wikipedia.org/wiki/Mark_Gorton
Oh well.
I swear I think we're living in the episode of TNG with the brain worms every day now.
> Gorton is involved in various green lifestyle issues, especially those having to do with transportation. At one point, Gorton was the single largest supporter of Transportation Alternatives, the New York City-based advocacy group for pedestrians, cyclists, and public transit.[10] In 1999 he founded OpenPlans, a non-profit organization that developed GeoServer, a collaborative open source project encouraging green urban planning initiatives.[11] In 2009 Utne Reader named Gorton one of "50 visionaries who are changing your world".
Covid/Trump broke Democrats' brains.