/tech/ - Technology★

>It's about as safe as a torrent right now

no it's not. bittorrent uses sha1 which was shattered and has been deprecated decades ago.

where's the release that doesn't crash routers by holding 1500 connections open

>>793232

>his router cant handle >10^5 connections

lol

>>793232

I think they've fixed it.

>>793234

>running 100% of your internet traffic through post-2013 pozzed hardware with proprietary firmware

If gateways make IPFS accessible to everyone, why isn't it more widely used?

>>793255

because nobody uses it.

>>793255

Client is badly optimized right now, it takes loads of ram and CPU for what bittorrent clients can do without breaking a sweat, they haven't worked on optimization since the protocol is constantly changing.

>>793268

>the protocol is constantly changing

this is also why nobody uses it

stop breaking fucking links every 2 weeks

Why did you make a new thread? The old one was fine >>771999

>>793270

>stop breaking fucking links every 2 weeks

You have no clue what you're talking about. The only non-backwards compatible change they made was in April 2016 when they released 0.4.0. Since then all changes have been backwards compatible with new releases.

>>793282

It had no image.

>>793232

Launch it with

ipfs daemon --routing=dhtclient

to reduce the amount of connections it uses.

Additionally, some progress has been made limiting the extent of the problem, though the issue of closing connections doesn't appear to be touched yet.

Issues to watch on this subject:

https://github.com/ipfs/go-ipfs/issues/4029

https://github.com/ipfs/js-ipfs/issues/962

If you're going to make a new thread, at least post some updates.

js-ipfs 0.26 Released

https://blog.ipfs.io/30-js-ipfs-0-26/

>Here are some of the highlights for this new js-ipfs release. There were plenty more bug fixes, tiny performance improvements, doc improvements and others all across the js-ipfs module ecosystem. A really BIG THANK YOU to everyone that has been contributing with code, tests, examples and also bug reports! They help us identify situations that we miss without tests.

New InterPlanetary Infrastructure

>You might have noticed some hiccups a couple of weeks ago. That was due to a revamp and improvement in our infrastructure that separated Bootstraper nodes from Gateway nodes. We’ve now fixed that by ensuring that a js-ipfs node connects to all of them. More nodes on https://github.com/ipfs/js-ipfs/issues/973 and https://github.com/ipfs/js-ipfs/pull/975. Thanks @lgierth for improving IPFS infra and for setting up all of those DNS websockets endpoints for js-ipfs to connect to :)

Now js-ipfs packs the IPFS Gateway as well

>You read it right! Now, js-ipfs packs the IPFS Gateway and launches it when you boot a daemon (jsipfs daemon). With this, you can use js-ipfs to access content in the browser just like you use to do in go-ipfs or use js-ipfs as a complete solution to add content in the network and preview it without leaving JS land. It is great for tooling. This was an awesome contribution from @ya7ya and @harshjv who spent a lot of time adjusting and iterating on the implementation to make sure it would fit with the structure of js-ipfs, 👏🏽👏🏽👏🏽👏🏽.

Huge performance and memory improvement

>With reports such as https://github.com/ipfs/js-ipfs/issues/952, we started investigating what were the actual culprits for such memory waste that would lead the browser to crash. It turns out that there were two and we got one fixed. The two were:

>>browserify-aes - @dignifiedquire identified that there were a lot of Buffers being allocated in browserify-aes, the AES shim we use in the browser (this was only a issue in the browser) and promptly came with a fix https://github.com/crypto-browserify/browserify-aes/pull/48 👏🏽👏🏽👏🏽👏🏽

>>WebRTC - WebRTC is really cpu+mem hungry and our combination of opening multiple connections without limits + the constant switch between transport and routing at the main thread, leads to some undesirable situations where the browser simply crashes for so much thrashing. We are actively working on this with Connection Closing.

>That said, situations such as https://github.com/ipfs/js-ipfs/issues/952 are now fixed. Happy file browser sharing! :)

Now git is also one of the IPLD supported formats by js-ipfs

>Now js-ipfs supports ipld-git! This is huge, it means that you can traverse through git objects using the same DAG API that you use for Ethereum, dag-pb and dag-cbor. This feature came in with an example, go check out how to traverse a git repo. 👏🏽👏🏽 to @magik6k for shipping this in record time.

The libp2p-webrtc-star multiaddrs have been fixed

>@diasdavid (me) and @lgierth had a good convo and reviewed a bunch of stuff over Coffee ☕️, it was great!. During that chat, we figured that libp2p-webrtc-star multiaddrs have been implemented incorrectly and figured out the migration path to the correct version.

>You can learn more what this endeavour involved here https://github.com/ipfs/js-ipfs/issues/981. Essentially, there are no more /libp2p-webrtc-star/dns4/star-signal.cloud.ipfs.team/wss, instead we use /dns4/star-signal.cloud.ipfs.team/wss/p2p-webrtc-star which signals the proper encapsulation you expect from a multiaddr.

New example showing how to stream video using hls.js

>@moshisushi developed a video streamer on top of js-ipfs and shared an example with us. You can now find that example as part of the examples set in this repo. Check https://github.com/ipfs/js-ipfs/tree/master/examples/browser-video-streaming, it is super cool 👏🏽👏🏽👏🏽👏🏽.

>HLS (Apple’s HTTP Live Streaming) is one of the several protocols currently available for adaptive bitrate streaming.

webcrypto-ossl was removed from the dependency tree

>We’ve purged webcrypto-ossl from the dependency tree. It was only used to generate RSA keys faster (significantly faster) but at the same time, it caused a lot of hurdles to being a native dependency.

>There is an open issue on the Node.js project to expose the RSA key generation primitive, if you have a use case for it please do share it in that thread. This would enable js-ipfs to use the native crypto module, have the same (or better) perf of webcrypto-ossl and not have to deal with an external native dependency.

PubSub tutorial published

>@pgte published an amazing tutorial on how to use PubSub with js-ipfs and in the browser! Read it on the IPFS Blog https://blog.ipfs.io/29-js-ipfs-pubsub.

do ipfs still use the ipfs (((package))) app that

throws away the cryptographic integrity of git?

do ipfs still use the ipfs (((package))) app that

throws away the cryptographic integrity of git?

>>793365

>>793366

NixOS will have IPFS as well WEW

this is nice but i can do without the pedoshit links

Call us when it can use both i2p/tor. Until then, it's inferior to bittorrent for filesharing. Why? Because the only client availables (https://github.com/Agorise/c-ipfs isn't ready enough) are in GC using retarded langages.

>>793478

They're working on it. c-ipfs seems promising as well.

>>793450

What pedo links?

>>793232

Turn on IPv6 you dumb nigger. NAT is cancer.

Does IPFS need the equivalent of TOR firefox?

My quick IPFS indexer seems to be working good, will try to make database available over IPFS if it keeps working good. Could anyone who knows how ES works download the 200gb of dumps from ipfs-search and run `grep --only-matching -P "[0-9A-Za-z]{46}"` on it? They're compressed/encrypted somehow.

>>793578

Nah, it's non-anonymous right now anyway, you could just install a new chromium/whatever and limit it to localhost:8080 if you really want to.

Can the anon behind this comment on whether js-ipfs 0.26 is enough to get it started?

https://github.com/DistributedMemetics/DM/issues/1

>>793676

I'd rather put my hopes into the actually-existing ipfs ib made by another anon here over anime-pic-one-commit over there.

>ipfs add 60MB in three files

>router reboots

This is 16chan-tier software

>>793991

Now is not the time for optimization, that comes later.

Also, see >>793346

Why does IPFS idle at like 25% CPU usage and high ram usage? Its not even seeding often or downloading anything.

>>794006

How much later? They've known this shit is unusable for two years.

>>794395

Better spec, better design, and better future-proofed.

Problem is, right now it still send a fuckton of diagnostics information and is poorly optimized.

They're making progress, but it's not fast enough.

>>794395

It's not a honeypot where every post you make on any site is tracked globally and anyone can insert arbitrary javascript (automatically run by all clients) (which must be enabled completely at all time to view sites in the first place) so long as any content of any type is allowed for upload in any page where that content would appear, for one.

>>793478

bittorrent is NOT secure over TOR

>>794538

It's just as secure as any other protocol. Some clients might potentially leak your IP, but there is nothing inherently insecure about the protocol.

>>794577

Try it yourself. The steps are as follows:

- go to an arbitrary site

- upload whatever they allow, content is irrelevant

- you will find a local file that was created with the content you uploaded

- edit it to insert arbitrary content

simple as that, bypasses all sanitation attempts, etc.

There has been proofs of concepts on 0chan in the very early days of zeronet, and this has never been addressed. The zeronet devs seem to not care about any of the components that make this possible.

>>794597

sasuga redditware

This is all well and good but the problem with the internet today is that it relies on a centralized infrastructure to gain access to which one has to pay fees to people with the power to cut one off completely if not control the content being served.

Anyway,

QmdqDLpEKd7zJ5fHNv8r3a4vJVcbgT3g3yW2vqvVHHmKXk

>>793208 (OP)

>>Is it a meme?

>You be the judge.

>It has implementations in Go (meant for desktop integration) and Javascript (meant for browser/server integration) in active development that are functional right now, it has a bunch of side projects that build on it, and it divides important parts of its development (IPLD, libp2p, etc) into separate projects that allow for drop-in support for many existing technologies.

i have judged

>>794365

Try upgrading from a Pentium 3.

Also centralized web servers were conceived because of a very important flaw with peer to peer infrastructure

>guy who owns file turns off machine

>file is gone

>>795152

but you've got that wrong, a centralized server shuts down no one can ever access that file.

>>795152

Did you ponder a bit before typing such a sloppy mess? Those centralized cluster of web servers can still represent a peer in a p2p infrastructure. Nothing is forbidding such a cluster from joining the swarm and sharing files. Compared to a distributed web, centralization offers very little benefits in return and is only still around because it offers more control (to the owners of the files and servers).

>>795152

This is literally the same problem with web servers. Even your VPS is on an actual server. So if you're saying P2P (and self-hosting) has flaws well

>If the guy who owns AWS doesn't like you, no one can ever access that file.

>>795375

>>795384

>>795387

none of you are wrong but you've all missed his point

>>795403

What point, that you can't get a file if the only peer in the world that has it turns off his PC? Well no shit captain obvious, but that isn't a problem with the p2p infrastructure, that's a problem of people not giving enough of a shit to seed that file. Maybe Filecoin is a better solution to that, but then again who knows what will happen in the future.

>>795413

you talk too much man

>>795429

are you asking for an entry level explanation of how it works?

>>795431

no, how would you apply p2p to a traditional website.

also what i dont like about ipfs is that it's almost impossible to have any privacy or remove content.

>>795432

>no, how would you apply p2p to a traditional website.

decentralized hosting, I'm not sure I understand the question?

>>795434

im talking about the code, how would you apply p2p.

>>795436

that depends on the type of site anon, come on

>>795438

well, let me make a mockup.

>>795441

that's almost exactly the same as IPFS chan

>>794597

sorry for asking to be spoonfed, but do you mind showing me a bug about this? I can't find it.

Based on what I can tell, you can't just arbitrarily change 0chan to post whatever file you want.

This isn't related to ipfs, so I'll sage

>>795423

Anon's just havin' a giggle, go ahead and click it.

>>794365

Unoptimized, probably the DHT.

>>794379

They're working on other stuff right now, they got a lot of money from filecoin ICO so we should be seeing some progress pretty soon. There's also a C implementation in the works.

>>794926

IPFS can run on any transport (hyperboria), which can run over any link (ronja)

>>795152

You cache it automatically when you download it.

>>795432

Tor integration is in the works for privacy

>>795441

>create keypair

>use this as board name

>sign admin keys

>they sign lists of posts to filter

>posts which do not follow specifications (eg too long text) are filtered by clients

>threads which are too old are filtered by clients

>>795441

You're mostly just describing smugboard >>785171, it's very similar to what you're proposing.

>>795446

It's not a bug. It's how it is designed. The poster can arbitrarily change the content of what they have posted. This includes changing the media type, and is simply a matter of editing the content that is stored locally. When someone requests the file, because you are the poster of the file, your doctored copy is distributed because the content you "upload" (e.g. text posts, or actual document attachments, etc.) are handled in this way.

You can even see the instructions on "how to modify a zeronet site" here:

https://github.com/HelloZeroNet/ZeroNet

as comments you post to a site are not handled in any special way compared to anything else. It's also why you need an ID to post anything and why your ID can be used to track anything you say across all sites by a simple grep: to enable modifying the content (which is not differentiable from a site, up to a point) by signing a more recent copy of the content.

>>795442

wew, its worse than I thought

>>795064

but Tor works easily on a P3, why is IPFS special?

>>796092

IPFS is still in alpha (not optimized yet) and has the overhead of a complete p2p system (routing). Tor is much simpler to implement since every peer is not a contributing node. A large number of peers connect to a limited amount fast nodes. In IPFS every peer is also a node. This is the difference betwen Tor's decentralized network approach and IPFS's distributed network approach.

>>795866

IPFS uses a distributed naming system (ipns) to point to the latest version as well as static pointers (ipfs-based addresses) to point to specific files. This is to enable tracking the latest version (i.e. enable the ability to update content) while still giving the guarantees there was no tempering by the controller. Zeronet doesn't seem to care at all about such possibilities: all that matters, only about the ability to update the content. Similarly, the zeronet folks don't give a shit about security (for the longest time (that might still be the case) they had been running with very old versions of various libs, including crypto libs, with unaddressed CVEs, for example). You can just say "their threat model is different" but at this point they disregard secops 101.

Release candidates are out for go-ipfs 0.4.11. If you want to try them out, check out the download page: https://dist.ipfs.io/go-ipfs

If you have troubles with IPFS using way too much bandwidth (especially during add), memory leaks, or running out of file descriptors, you may want to make the jump as soon as possible. This version includes prototypes for a lot of new features designed to improve performance all around.

https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md

So if I use something like IFPS to host a website, does that mean that I don't have do fuck with things like domain name registration?

>>797401

Technically yes, but in practice you will still need a way to register a user-friendly name because people can't recall localhost:8080/ipns/LEuori324n2klAJFieow. But there's a way to add a friendly name in the ipns system (google around, I don't recall the correct method), which allows people to use localhost:8080/ipns/your.address.name instead, so that's an option. Other than that, all kinds of systems can leverage the likes of namecoin if you're so inclined.

>>797461

Eh, I actually prefer the hash method. Keeps things a little more comfy.

>In the future, we may have ipns entries work as a git commit chain, with each successive entry pointing back in time to other values.

That's really fucking cool though.

>>797484

http://decentralized.blog/ten-terrible-attempts-to-make-ipfs-human-friendly.html

Here's a list of all DNS alternatives that IPFS team can use, my guess is that they will use Filecoin considering that it belongs to them.

>>797484

The method is to register with any normal DNS method a TXT record with content: dnslink="/ipns/<hash>" and it will work. So it's actually relying on the external system.

>>797487

I thought filecoin was just an incentive to store other people's files?

>>797541

>I thought filecoin was just an incentive to store other people's files?

It is. I think they're going to recommend using ethereum domains as IPFS has plans to be deeply integrated with it.

>>797839

>Can I limit the amount of space that IPFS uses

IPFS doesn't download random things to your computer. It caches everything you view but by default it's capped at 10GB.

>>797839

By default IPFS does not fetch anything on its own, it only will retain the data you manually added via browsing or manual adding.

If you want you can run the daemon like this `ipfs daemon --enable-gc` which will read your config for 2 values, 1 is a timer and the other is storage. By default I think they're 1 hour and 10GBs, that means a gabrage collection routine would run either when you hit 10GB's of garbage or 1 hour has passed. What it considers garbage is anything that's not "pinned", if you don't want something to be treated like garbage you pin it.

Someone made an issue recently that I agree with, there should be an option for a minimum amount of data to keep, right now garbage collection deletes ALL garbage, but it would be nice if you could set it to keep xGB's worth of non-pinned content at any one time.

https://github.com/ipfs/go-ipfs/issues/3092

>>797863

Have you upgraded to 0.4.11 yet?

I have a question about implementation

Each file is divided into chunks, which are then hashed. These hashed chunks form the leaves of the merkle tree, which have parents that are identified by HASH( HASH( left-child ) + HASH( right-child )). This continues until we reach the root node, the merkle root, whose hash uniquely identifies the file.

To give someone else the file, from computer S to computer T, S gives T the list of leaves, and the merkle root. As I understand it, this is basically what a bittorent magnet link does as well (along with tracker and other metadata). We know the leaves actually compose the merkle root, by simply building the tree from its leaves, and verifying the new merkle root is the same as the provided one.

Computer T then ask around if anyone else has the content of the leaves (by querying for the leaf-hash), and verifies the content by hashing it upon download-completion. Once it has everything (and verifies), it simply compiles the parts into the

Assuming there is nothing wrong with my understanding above, I have a few questions:

How do we know the merkle root actually identifies the file we meant to get? ie if someone hits an IPNS endpoint, and an attacker intercepts and returns a malicious merkle root + leaves, now what? Is there anything to do about this or is this just a case of don't trust sites you don't know

When computer T starts requesting for leaf-content, is it requesting by querying on the hash of a leaf, or the merkle root? Bittorent only requests parts from users that have the full file, which comes from the latter. If you request by the leaf-hash instead, I'm imagining that the less-unique parts (like say a chunk of the file composed entirely of NULL bytes) could come from ANY file source, regardless if that user actually has the file you're looking for.

And extending that, with some infinite number of files stored globally, it would be possible to download files with a leaf-list that NO ONE actually has; each leaf being found in some other file; composed in some particular fashion to create the requested file.

Can you use IPFS in combination with a tor bridge with obfs encrypting files and still transfer files? If this worked would the person receiving the data still see your public IP?

>>797872

>How do we know the merkle root actually identifies the file we meant to get?

So you mean how is the data verified to be correct once the client receives it. inb4 is isn't verified

>>797872

>When computer T starts requesting for leaf-content, is it requesting by querying on the hash of a leaf, or the merkle root?

On the leaf. Each 256k block has its own DHT entry (hence why it's known to be so chatty). This also means that if you have a file and change one byte in it then most of the file will be deduped by IPFS if you readd it.

>>797872

>if someone hits an IPNS endpoint, and an attacker intercepts and returns a malicious merkle root + leaves, now what?

My understanding is that the IPNS entries are signed by your public key, so that's not an issue. There is a problem where a malicious node could return an old entry, but that's the reason each entry is stored in a fuck-ton of DHT nodes. Which is also the reason it takes so long to resolve names, it doesn't just take the first resolution it can.

>>797878

>On the leaf.

So what I suggested then, that a file no one has could be generated by the network given a list of leaves, by retrieving them from other files, would hold then? I suppose though that's not anything special, except that the granularity of chunks is bigger than say, 1 bit. But to confirm my understanding, is this true?

>>797845

With bitswap, it DOES download random things, kinda. You swap fragments among peers from random content.

>>798003

It's hash-based addressing: if the chunks that make up the file exist in other files, they are exactly as valid in the requested file as it is in that other file. That is, request the hash and the provenance is a meaningless concept: you can think of it as two completely different kinds of data (the actual chunks, and the file descriptors which are merkel graphs)

>>797872

> ie if someone hits an IPNS endpoint, and an attacker intercepts and returns a malicious merkle root + leaves, now what?

IPNS is still handled via DNS, so short of someone pwning the authoritative nameserver for a domain, you're looking at a hijacked local resolver, which you can defeat via VPN.

updated my porn folder again

/ipns/QmVm4jMdZnewAAU3QPoUBJ6jpjjicRWsfcjfD7c47rf1KC/latest.html

>>798921

direct link since ipns is buggy

/ipfs/zDMZof1m2wGAGywacnVpmTXZ76tW4EWixSdVz1rkkNGLj3d5vAuh/

alright my first try at this, it's Lovecrafts "beyond the wall of sleep"

/ipfs/QmShe7riU5RVJ7iGkr7ebMqsgUjMk5SfiqeRMeB1Hnu6gX

>merkel tree

/pol/ shoop incoming

>>798921

>>798927

Nobody is interested in the contents of your spank folder, you degenerate.

>>799218

but someone might be

>>799291

You can check yourself by accessing a file through the gateway, i.e. https://ipfs.io/ipfs/QmT45tFQo5DJ8m7VShLPecsTsGy1aSKBq4Pww8MfaMppK6

If you can see it there, everyone can find it.

>>798927

>ipns is buggy

How do you mean?

>>800028

ipns forgets what it's linked to after 12 hours and is extremely slow

https://podricing.pw/posts/1191416

>Right now if you are Catalonian you can go and look up where to vote for the independence of your country via an encrypted and fully distributed web running on experimental, unfinished free and open source software maintained by a global network of volunteers, which is the only way to do it since the government censored the original website and is sending the cops after everyone who tries to mirror it normally. The future is now, motherfuckers.

>>793487

RIght near the top of page linked by OP is the following:

gathered links. mostly from IPFS generals on 8ch/tech/.

* anon's qt ipfs page (this site)

/ipns/Qmeg1Hqu2Dxf35TxDg18b7StQTMwjCqhWigm8ANgm8wA3p

* the best cp archive i've ever seen

/ipfs/QmY7KEmJKpx7bNDQ2WfDJp2zdsvX1ATZKWd4AXAhDLCaBM

I ain't clicking it, regardless of what it may hold. Probably cartoon ponies teaching how to circuit-probe.

>>799291

>And here the necronomicon

>could someone tell me if it works alright?

No one can *tell* you that magick works, Anon. You have to say the incantation yourself. If you see results that seem magical to *you* then the magick has worked for *your* belief. Magick is not science.

>>800158

Disclaimer: I think this is all right but I'm not sure since things are changing rapidly, correct me if I'm wrong

Nodes don't forget, the record expires, by default your daemon is supposed to re-publish anything you published every 12 hours and records expire every 24

https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#ipns

Just make sure you actually publish after starting the daemon and again anytime the daemon restarts. I think this will be automated later and other peers will keep the record alive but it's not like that yet.

>>800207

It's Rick Astley's Never Gonna Give You Up

>>799291

Are you still hosting? I managed to get only 25MB's.

>>800209

If magic only exists based on personal perception then magic cannot affect other people in any way, since their perception differs from yours.

>>800287

yeah I forgot to turn it on, try again.

IPFS v0.4.11 is out

This looks like a large update. Everybody should upgrade.

>You will now be able to configure ipfs datastore to use flatfs, leveldb, badger, an in-memory datastore, and more to suit your needs

>The concept of 'Bitswap Sessions' allows bitswap to associate requests for different blocks to the same underlying session, and from that infer better ways of requesting that data

>As nodes update to this and future versions, expect to see idle bandwidth usage on the ipfs network go down noticeably

>Users who previously received "too many open files" errors should see this much less often in 0.4.1

>A memory leak in the DHT was identified and fixed and now memory usage appears to be stable over time

>In an effort to provide a more ubiquitous p2p mesh, we have implemented a relay mechanism that allows willing peers to relay traffic for other peers who might not otherwise be able to communicate with each other

>we have come up with a system that allows users to install plugins to the vanilla ipfs daemon that augment its capabilities (git and ethereum plugins are currently available)

>In the future, we will be adding plugins for other things like datastore backends and specialized libp2p network transports

>we've switched the ipfs/go-ipfs docker image from a musl base to a glibc base

https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md

>>800335

>going from muslc to botnetc

Hmm. As long as you are compiling it yourself still......

>>800335

2 weeks old my man.

>>793208 (OP)

>It's about as safe as a torrent right now

>>793223

>no it's not. bittorrent uses sha1 which was shattered and has been deprecated decades ago.

Data on the attack. https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html

For the time being, it's still not considered a realistic scenario because of the amount of effort involved, even if it's a possible one. Either way, they've added SHA256 to the BitTorrent v2 specifications.

http://bittorrent.org/beps/bep_0052.html

>>800423

>Either way, they've added SHA256 to the BitTorrent v2 specifications.

Fucking finally.

>>800387

It was released the 27th. You're thinking of the release candidate.

>>800423

>SHA2-256

>not using SHA2-512/256 or BLAKE2b-256

That was either a retarded decision or a (((perfectly planned))) decision. Why would they slow down the entire network to use an algorithm that isn't even recommended anymore due to the prevalent security threats? And don't give me any muh hardware acceleration crap. The only x86 CPUs that have hw acceleration are newer Intel Atoms and AND Zen which count for a minuscule percent of torrent users.

>>800544

It was (((perfectly planned))) considering the (((endurance international group))) owns bittorrent now.

>>800544

>And don't give me any muh hardware acceleration crap.

That's exactly the excuses I was reading in the Issue thread on the subject. It sounded stupid to me too.

>not using SHA2-512/256 or BLAKE2b-256

BitTorrent is an open standard. It's up to the community to implement. If the community got together and decided to implement a modified protocol, the recommended specs would necessarily have to change to reflect this if enough people were about it. That or we could fork and call it ButtTorrent, since we're being stubborn butts about it and everyone just calls it torrenting anyway.

>>800207

Those titles were a joke, nigger

Do you think it will work in the outer space?

>>804662

No.

>>804804

how would you prevent another tay?

>>804927

>how would you prevent a large corporation from deleting its chatbot after we ask it to repeat 14/88 memes?

I don't think there's a solution to that other than do it yourself. If there's source code, you can rebuild it; if there isn't, tough luck. I don't see what IPFS has to do with this, other than maybe helping to host it or its databases.

To whomever posted QmVuqQudeX8dhPDL8SPZbngvBXHxHWiPPoYLGgBudM1LR5 and QmT45tFQo5DJ8m7VShLPecsTsGy1aSKBq4Pww8MfaMppK6 these stop part-way. Please continue to seed them.

>>804994

>>804997

Okay I'm up now, badger seems fast as heck compared to flatfs.

I made a mistake earlier, I only have 25% of /ipfs/QmT45tFQo5DJ8m7VShLPecsTsGy1aSKBq4Pww8MfaMppK6

I knew I had 100% of /ipfs/QmVuqQudeX8dhPDL8SPZbngvBXHxHWiPPoYLGgBudM1LR5 though

OP should come online so I can finish the mirror.

>>805334

Are there any benchmarks on datastore filesystems yet?

>>807340

The benchmarks for the underlying systems should be representative enough. There are plenty online of people comparing leveldb with other things, badger itself competes with an improved version of leveldb(RocksDB) and the devs posted a lot of information about both of them in their post, they also have comparison results.

https://blog.dgraph.io/post/badger/

There's probably no benchmark for the flatfs scheme the ipfs team made, I don't think it would make sense to bench either since the underlying filesystem and OS have a major impact on it, since it's just flat files in directories.

Reminder that the interface for datastores is in place and currently being worked on, so you can implement the interface and use whichever datastore you want. Most of (if not all of) the filestore(no-copy adds) work was done by a third party.

>>808110

I suppose badger is a no brainer considering they're making it the default next patch, as far as I can tell.

>>808994

It seems objectively better than leveldb in every way, personally it's been much better than leveldb and flatfs on my machine, like noticeably so in terms of speed, hashes went from taking about 2 seconds to be fetched on the public gateway, to instant, and this is with hundreds of gigabytes of things in my datastore.

>>808995

Surprising how much overhead there is in flatfs just from storing everything as 256kiB files. Makes me wonder how much faster git would be with an sqlite backend.

bump for interest

>>813377

What are you interested in?

>>813388

they meant usury

>>793208 (OP)

>thinking anyone will use this

>thinking it won't get outlawed

>>813850

Many people do use this without even knowing, dumb fuck. Get out, candy ass piss ant; Inform yourself before speaking. This isn't some inconvenient technology. On the contrary, very fast and improves speed over regular internet usage. This isn't even addressing the other benefits. BTFO.

>>814476

Whoa, calm down there sis.

>>813850

Outlawed or not it would be hard to prevent people from using it. The development team is coming up with strategies to use it even in restrictive environments.

https://github.com/ipfs/go-ipfs/issues/3908

At best you'd be able to prevent some part from working easily but since the plan is to be able to swap out components, you'd still be able to utilize it by getting around blocks. Things like cjdns and message passing will make it very difficult to prevent data access.

Version 0.4.12 has a release candidate out now. Here's part of the changelog:

>Ipfs 0.4.12 brings with it many important fixes for the huge spike in network size we've seen this past month. These changes include the Connection Manager, faster batching in ipfs add, libp2p fixes that reduce CPU usage, and a bunch of new documentation.

>The most critical change is the 'Connection Manager': it allows an ipfs node to maintain a limited set of connections to other peers in the network. By default (and with no config changes required by the user), ipfs nodes will now try to maintain between 600 and 900 open connections. These limits are still likely higher than needed, and future releases may lower the default recommendation, but for now we want to make changes gradually. The rationale for this selection of numbers is as follows:

>>The DHT routing table for a large network may rise to around 400 peers

>>Bitswap connections tend to be separate from the DHT

>>PubSub connections also generally are another distinct set of peers (including js-ipfs nodes)

>Because of this, we selected 600 as a 'LowWater' number, and 900 as a 'HighWater' number to avoid having to clear out connections too frequently. You can configure different numbers as you see fit via the Swarm.ConnMgr field in your ipfs config file. See here for more details.

>Disk utilization during ipfs add has been optimized for large files by doing batch writes in parallel. Previously, when adding a large file, users might have noticed that the add progressed by about 8MB at a time, with brief pauses in between. This was caused by quickly filling up the batch, then blocking while it was writing to disk. We now write to disk in the background while continuing to add the remainder of the file.

>Other changes in this release have noticeably reduced memory consumption and CPU usage. This was done by optimising some frequently called functions in libp2p that were expensive in terms of both CPU usage and memory allocations. We also lowered the yamux accept buffer sizes which were raised over a year ago to combat a separate bug that has since been fixed.

>And finally, thank you to everyone who filed bugs, tested out the release candidates, filed pull requests, and contributed in any other way to this release!

>>818479

The final version of 0.4.12 was released yesterday. Now that it won't spam my network as much I might host some content. But the real question is, is my bandwith is better spent seeding the torrents or seeding them on IPFS?

>>818544

>torrents or IPFS

Do you think it's likely you'd max out with both running? I tend to have my torrent and ipfs daemons running simultaneously, if I get bittorrent traffic it's usually just a burst for an hour or a very slow trickle to someone on the other side of the world. Same with IPFS, most of the time it's idle, and I have a lot of content shared on both + other networks.

Right now I have both running + soulseek.

>>793208 (OP)

>communism: the protocol

>>818576

A global network is something I'm okay with my computer contributing to. It's robot communism at best. The costs and benefits of a decentralized and/or distributed system are more appealing than the faults associated with a centralized system, in my opinion.

The details of IPFS itself (while not new) seem especially nice, things like immutability, content addressing, and the inherent lack of trust associated with P2P systems which encourages better validation and security practices at a network level.

Things like pinning services (essentially CDNs), private networks, and real-time dynamic content via pubsub, allow for some opportunity for capitalists as well.

I think things like Freenet which force you to share the load are more communistic, with IPFS you're only ever sharing what you yourself want to share.

Wew, page 9. Anyway...

Charlie Manson Superstar (ogv)

/ipfs/QmdSCNSSdpS3j6vudHR87FChHHus4jsgMKGMKLP4g86tRM

>>818576

What a quality argument. It's good to know that the best and brightest are still here.

go-ipfs 0.4.13 is out. Judging by how quick it was, you should probably download it immediately if you're on 0.4.12 already.

>Ipfs 0.4.13 is a patch release that fixes two high priority issues that were discovered in the 0.4.12 release.

>Bugfixes:

>Fix periodic bitswap deadlock (ipfs/go-ipfs#4386)

<If a sessions context gets canceled during an isInterested request, bitswap (all of bitswap) can deadlock.

>>Fix badgerds crash on startup (ipfs/go-ipfs#4384)

<After changing the datastore to badgerds you can no longer start a daemon without a panic.

oh hi i hrd thr ws cp?

https://github.com/ipfs/js-ipfs/issues/1095

>>829212

Nothing very interesting there.

>>837235

Is your daemon running? I can't access this.

also prefixed for the browser addon /ipfs/QmbNZZpQRThNurXNPhcazA2Gw52FCC3SxChiC3T6GiMB3e

Saving the thread while we wait for a new version, so I might as well ask: What is the difference between pin and add? I'm still not really clear on that.

no offense guys but.. why care about this shit? I can't use it through tor or i2p so it's just the same as torrents: SEND ME A DMCA PLZ! I want 3 strikes and my net cut off!

>>849031

Pin tells the GC not to reap the blocks until you unpin them.

>>849031

>>849617

Reminder that by default, adding something pins it, if you don't want that use

ipfs add --pin=false

ipfs pin add *hash*

So when are we expecting for DHT to work in js-ipfs, so we can properly run it on websites?

At the moment, running a node on a webpage only allows you to retrieve content that the default gateways seem to have stored.

>>850390

Would it be possible to just continuously reannounce to keep the files cached in the gateways? It's a shitty solution but it might work as a stopgap until js-ipfs is more complete.

bump

>>849036

You are the ultimate brainlet. This isn't just for pirating shit, it's also for censorship circumvention and freedum.

Where is my in-page IPFS without gateways? Once we have that it will be a new age of Internet prosperity. Smugboard can finally replace this crumbling shell of an imageboard.

>>851480

>Where is my in-page IPFS without gateways?

Eventually. It seems js-ipfs doesn't yet have proper routing or something, so it can't connect to the DHT, and or connect to nodes it hasn't already connected to. This limits js-ipfs to any node on the bootstrap list.

I was thinking it'd be fine to just use go-ipfs and have people download a client (which runs the daemon in the background), but seems like everyone these days wants to do everything from within the browser. Thus, we wait for js-ipfs to reach a usable state.

>>851484

>Thus, we wait for js-ipfs to reach a usable state.

Yes, don't wanna blow our load before making it perfectly transparent for normies. They should use those filecoin shekels to hire like 20 more devs though, is this not the most important development in Internet history since social media? Where are the buzzwords, the hype? Oh right, this will kill CDNs and file hosting sites (jewtube etc) among other things.

>>851485

>They should use those filecoin shekels to hire like 20 more devs though

I agree. Maybe I'd see it if I really analyzed how many people are making pull requests, but it seems like things are moving the same speed they always did. Granted the big things to tackle moving forward are executive decisions about how to implement complicated shit like bitswap, but you can pay people to help with bug fixing while you approach conceptual solutions. Filecoin can't live without IPFS and vice versa. It's very much a yin-yang thing. I'd like to see some serious muscle put into shipping out a 1.0 before Filecoin even hits open beta. God forbid they come out of the gate with critical bugs or scaling issues.

Another nice thing to see would be hiring someone to help develop community projects. Imagine a guy who knows the software back to front helping out all the little guys in that IPFS Shipyard, especially things that could work as building blocks for bigger projects.

>>850390

When https://github.com/ipfs/js-ipfs/pull/856 is merged. Which is taking fucking forever.

>>851561

>it feels like the community understands this too

This is true, it's pretty clear to me everyone seems to know what's up regarding this. Nobody wants to push this mainstream until it's something we can stand behind.

Someone want to explain to me how the

ipfs p2p

If I'm understanding correctly, you can basically listen in for TCP/UDP connections, and connect to them by resolving an IPFS Peer-ID, instead of a domain or IP address?

Looks pretty useful; has anyone actually made use of these features?

>>851910

It's experimental, so documentation is light. This thread has an easy example demonstration: https://github.com/ipfs/go-ipfs/issues/3994

>To enable the command, you will need to run:

<ipfs config --json Experimental.Libp2pStreamMounting true

>And restart your daemon.

>Basic usage of ipfs p2p is as follows:

>First, open up a p2p listener on one ipfs node:

<p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101

>This will create a libp2p protocol handler for a protocol named p2p-test and then open up a tcp socket locally on port 10101. Your application can now connect to that port to interact with the service.

>On the other ipfs node, connect to the listener on node A

<ipfsi 1 p2p stream dial $NODE_A_PEERID p2p-test /ip4/127.0.0.1/tcp/10102

>Now the ipfs nodes are connected to eachother. This command also created another tcp socket for the other end of your application to connect to. Once the other end connects to that socket, both sides of your application should be able to communicate, with their traffic being routed through libp2p.

>The easiest way to try this out is with netcat.

I'm not sure what he meant by "ipfsi 1" but I assume it's a typo, and you should just change that to "ipfs".

>>851917

Seems more fleshed out for js-ipfs, for reasons that are only natural.

https://github.com/libp2p/js-libp2p/tree/master/examples

>>851971

libp2p is a set of protocols and utilities, of which IPFS utilizes most of them. "ipfs p2p" is a subcommand for binding a normal application to your PeerID via ports, so it's possible to create a centralized website on top of IPFS, with decentralized addressing.

>>851976

Yeah, nevermind, it's pretty much the same thing, it looks like.

https://github.com/libp2p/go-libp2p/tree/master/examples

Or at least, it's a part of it.

>>852424

They have to do that on the main gateway to avoid getting vanned for knowingly hosting illegal content. If you want to link it to other people who don't have ipfs, you can use someone elses gateway or better yet, use your own with your own blacklist.

http://ipfs.git.sexy/sketches/run_a_gateway.html

I never even looked into how to setup a blacklist for it though.

What's the content of that hash?

>>852436

Found it on /ipfs/QmU5XsVwvJfTcCwqkK1SmTqDmXWSQWaTa7ZcVLY2PDxNxG/ipfs_links.html

It's labeled as "Programming books".

>>852450

Sipser's book isn't that bad. What are you getting at anon?

>>852450

I had high hopes after the first category, which is spot on.

If your school teaches Java, it's basically a "Java school", there is literally no lower circle of hell for a CS school, except teaching C# maybe.

Second category : Papadimitriou is a respected computer scientist, was this book that bad?

Third category : Sipser taught at MIT for 30 years.

Fourth category : comparing apples and oranges, I very much doubt that books on the left are used at the same level or for the same course as books on the left.

I will do my own list, cool idea.

http://ipfs-search.com

Seems to be up again.

>>855725

>How can I 'seed' content? Download and 'add' the directory later? I am not seeing an obvious answer.

Yeah. ipfs get will pin it temporarily to your datastore, but doing an ipfs add or ipfs pin after will hold it indefinitely. I'd recommend moving it to a permanent location on your drive and using add so you can use the --nocopy flag to save disk space.

>Any pointers in how to use it? Couldn't find ditshick translations with it.

I don't know how long IPFS Search has been up, so they might show up after another update or reannounce.

>>852573

>C++

Good schools use the K&R, ANSI version.

>>856480

I think if you poke around .ipfs/config you can choose how you connect but I'm not really sure. However, you shouldn't be running IPFS through TOR in the first place.

>>855821

>ipfs get will pin it temporarily

Stuff like this will probably confuse people, `get` doesn't pin anything, it just caches the content in the datastore. Pinning content just takes cached content and flags it so that the garbage collector does not reap it.

>save disk space

Another option is mounting ipfs and using the mfs if you want to reorganize the layout.

>>856480

https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#addresses

Make sure you're just listening on the tor interface.

Also look into this guys fork so you can just connect to tor directly instead of proxying.

https://github.com/ipfs/notes/issues/37#issuecomment-247717781

If you go that route you can put just that in your listener section.

nsa honeypot

What sets this project apart from freenet? From what I've read so far, it seems very similar.

We're watching you, non-believers~

What happens if someone uploads cp? Is it just on everyone's computers forever?

>>861400

Why can't you people read the thread, it doesn't distribute implicitly, if someone hashes CP it's only on their node unless you download it too, if you download it on accident just run garbage collection or wait for garbage collection to run automatically if you have that set.

>>861400

Indeed. And Lord Google shall pin it as the darkness, and the masses shall follow Their word.

>>861399

As does all of /tech/, possibly even all of 8ch!

>>861401

Oh, okay.

What a useless protocol, then.

>>861404

You'd rather it be stuck on your node? What's the benefit?

>>861405

I expected some protocol-layer encryption

>>861407

There is, that doesn't have anything to do with automatic distribution across nodes. I'm not sure how you mixed those up.

/ipfs/QmYShhhD6j7vwL4SGUp2vNwHerRc7VU4Qm86ZJWvgqZn9G - ReleasetheMemo.pdf

/ipfs/QmbtL1GXPaCwUd1k8iBHynpGZUM6CwrLAuHR9LjZTtcUYB - Damore Complaint against Google, 1-8-2018.pdf

/ipfs/ QmUUA8zudreYSSoGp8aujw1d6QJAjoan9W7MATF7udjrws James_Mason_-_SIEGE_3rd_Edition.pdf

http://inclibuql666c5c4.onion.link/

Someone should IPFS the books in here.

These people need it the most (especially the Rabin fingerprinting)

>>>/pdfs/ >>>/tdt/ >>>/zundel/

https://github.com/ipfs/archives/issues/137

https://github.com/ipfs/notes/issues/1

>>863660

>@rht You're violating our code of conduct. This is not the first time.

>I'm not willing to tolerate being bullied in this way and I completely disagree with your repeated, baseless implications that people are being dishonest or trying to hide information. We make huge efforts to operate in transparent ways and to provide people with good information. There are plenty of ways to press for better information without falsely and baselessly accusing people of bad intentions. We want everyone to help us make sure that we're providing the best possible information in transparent ways, but you must also respect our need to maintain a Friendly, Harrassment-free Space.

Good luck with your faggotware.

>>863669

>When faggotware is better than torrents

>>863660

Stop trying to get more people to use it right now, it's not finished yet. It's good for us because we know how to use it. Promoting it to less tech savvy users isn't going to help anyone and looks like spam.

Rabin isn't even out of experimental yet and the only browser that supports IPFS as a valid protocol is Firefox 59. Wait until it's finished before telling other people to use it.

>>863669

The guy telling him off (flyingzumwalt) doesn't do anything important, he schedules the meetings and is resident bitchboi on the forums. The real project maintainers seem alright.

>>863288

Why don't you do it? If you post the hash I'll pin it.

https://github.com/ipfs/archives/issues/87 # doing a "sprint"

https://github.com/ipfs/archives/issues/136 # test result

https://github.com/ipfs/archives/issues/137 # more results

https://github.com/ipfs/archives/issues/142 # wanting people to refute him

So rabin fingerprinting is worse than Gzip, and is comparable to classic chunking.

>>867327

Are there better alternatives than rabin and whatever IPFS does by default? Maybe someone should propose them. For one time operations on big datasets, having a slow but space efficient chunker/de-dupe system seems like it would be valuable.

I noticed a lot of those tests are using the default rabin size too, IPFS allows you to specify the min max and average chunk size, I don't know how effective settings these would be versus the default

ipfs add -s rabin-[min]-[avg]-[max]

This is a problem I always see in traditional filesystems, ones that offer de-dupe are usually very taxing on memory and a lot of them just recommend using compression methods like lz4.

>>867330

lel

>>867426

When the optimal rabin chunking between txt, html and epub are different, it will be tough for IPFS to implement

https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/

>>863714

>Stop trying to get more people to use it right now, it's not finished yet.

Fucking this.

>>871039

fukken saved

>>871039

cat > Documents/pasta/pickleshit.txt

>>871061

>>871062

What was it?

>>864553

Come back online you fricken frick. I only got 78% and want to mirror this for you.

>>871232

>What was it?

An autistic pickle rick post.

controlled opposition

report and ignore

>>872683

What?

Pink Flamingos (1972) - /ipfs/QmZUr7Si5aLU4ua6tmdHhpdXKWdvwAndccMtKWpd3HGSzn

JS-IPFS v0.28

https://github.com/ipfs/js-ipfs/issues/1228

Soon

/ipfs/QmdNRFVKhieT8FpkxkppXMct7y9ZpLjZJpDKeZFjoeWnoz - Rules_of_the_Internet_2.0.png

SPREAD THIS LINK

OFFICIAL 8CHAN RULES

/ipfs/QmQgfZp9wWSq1QdyxBeKF2YuH8cGpQ5vy6B6mib23dLQ37 bsd_coc_discussion.mbox

I think I'm the only one contributing to this thread now. Since I'm doing hacky shit atm because internet got shut off, I am unsure if it's uploaded or not. Enjoy it tho.

/ipfs/QmRCbRdih2hr44uzrE6aViVEAncAQ9hD61wiUSwoKhWspU

>>881679

Obligatory

>Windows

Buncha Touhou OVA's from an old IPFS thread. I should be able to seed for quite a while.

/ipfs/QmSsepMbw1ASbcysMjFHwvSND5PMhThf9UEbYsUoBneAAn

>>881734

sorry but i dont use meme os's.

>>881679

What's the problem? Make a new environment variable called 'IPFS_PATH' and set the value to 'D:\whatever\you\want'

>>882080

because ipfs claims the environment is not defined.

>>882055

Then don't use meme software, you faggot. Go back to /g/.

How do I know if my uploads are being peered? No one is replying to my posts so I don't know.

>>883013

>ipfs dht findprovs <key>... - Find peers in the DHT that can provide a specific value, given a key.

This will spit out peerid's that are online and have/provide the full hash.

>>883028

Is there a way to list all peering files in this manner?

>>883037

I'd probably loop over the output of

ipfs pin ls --type=recursive

I'm not sure your endgoal though.

>>883046

My endgoal would to make a tree-view of all files I've uploaded, with the peers peering each file

Dresden - The Call of the Blood (1996) (Flac format, meta-date double-checked)

/ipfs/QmQqF6FJDp9F57aL57w8ArTNuvKNzCm43d9MDBWWo7iG3X

>>881550

I think the thread is more for talking about the protocol and implementations rather than sharing files like /v/'s "share threads". Otherwise I'd be posting content. For now I'm waiting to see the go-ipfs 0.4.14 release. It should have a lot of performance improvements from the developers but also with the help of Go 1.10 having its own performance improvements.

/tech/ should have a share thread of their own, see how /v/ does it >>>/v/14459689

Just post tech related content through any means, BT, IPFS, DDL, etc.

It would be good for trading data but also talking about protocols and shit. The retroshare threads could be merged into it.

>>883013

>No one is replying to my posts so I don't know.

More lurkers than posters on /tech/. I try to mirror hashes that are posted in these threads, sometimes people post neat things. Sometimes people post a hash and go offline forever though which is annoying.

>>883736

>Sometimes people post a hash and go offline forever though which is annoying.

this fucking this

>>883070

It would be great to see how many peers are sharing same files like torrents

My waifu songs...enjoy

/ipfs/QmadHXY6TXeSasoc6DZyBruEtV6gfF4F1oSRbi6VTVdfdw

/ipfs/QmcrnR5d8UXLxYE7uYxkT4pNwwChZbYietbxG5bvkvr1xB

/ipfs/QmTWXWo3B3TxvVAt2M5N67JqMhr94Qa7kQEQV5ZDvmBMvz

has anyone tried making a booru on IPFS? i know about hydrus network, but i want something you can access from the browser

>>883745

>how many peers have this individual file vs how many have the whole directory/node-root.

I meant how many peers right now are downloading from me

>>883745

>If I wait long enough someone else will do it for me.

kek

A true "hacker" movie

/ipfs/QmfCurXyGFq2nyVDPdMa9qYEgTcRxCDiWv5wotM7b8vftB

Wario World NTSC-U

/ipfs/Qmbki3KYmZ31rvQ8fPnNLXD5KnKS67ckgEi6wgPyhp8eoU

>>800826

>butttorrent

hey reddit, did your girlfriend peg you in the ass again?

>>883754

>has anyone tried making a booru on IPFS?

Also curious about this.

>>885378

>>883754

>>>/hydrus/7858

IPFS 0.4.14 is out

Main improvements are lower resource usage and pubsub IPNS

>disentangled the commands code from core ipfs code, allowing us to move it out into a separate repository

>fixed several major issues around streaming outputs, progress bars, and error handling

>provide an API over other transports, such as websockets and unix domain sockets

>With the new pubsub IPNS resolver and publisher, you can subscribe to updates of an IPNS entry, and the owner can publish out changes in real time. With this, IPNS can become nearly instantaneous. To make use of this, simply start your ipfs daemon with the --enable-namesys-pubsub option, and all IPNS resolution and publishing will use pubsub

>Memory and CPU usage should see a noticeable improvement

>upgraded hashing library, base58 encoding library, and improved allocation patterns [NEON/SSE/AVX/AVX2 for SHA2 and Blake2b]

>This release of ipfs disallows the usage of insecure hash functions and lengths

https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md

Man I seriously can't wait for this to take off, this is possibly the best we'll get off the current decentralization/blockchain memes.

>>888288

never ever

>>889330

"never ever" as in we are never going to escaoe the blockchain meme

/ipns/QmPZmiFdf7PTHqrHYXQPSHY5PdgQC62L7JPw4GcrmGYJ8e

(currently /ipfs/QmfALsokxXrTCgBAixMXDuxgVexy6ySP823TnrJ4kPmDeY)

Comprehensive Libbie fanart archive.

>>888252

>With the new pubsub IPNS resolver and publisher, you can subscribe to updates of an IPNS entry, and the owner can publish out changes in real time. With this, IPNS can become nearly instantaneous.

>This release also brings the beginning of the ipfs 'Core API'. Once finished, the Core API will be the primary way to interact with go-ipfs using go. Both embedded nodes and nodes accessed over the http API will have the same interface.

Oh thank fuck. Now if only it would stop killing my shitty new router.

Anyone else have trouble using

ipfs pin add <hash>

>>890258

> I want to pin some rather large folders without having to download them first

What you want to do is add them first and they will automatically be pinned.

ipfs add -r <directory>

you can do them together as well

ipfs add -r <directory1> <directory2> ...

And you can still add them and not pin them right away with

ipfs add --pin=false ...

When you go to add a pin it will take the parts you already have and download parts you're missing from other peers.

If it stalls it may mean you can't see any peers with the content, which can be a connection issue or just the fact that nobody is hosting it.

I tend to always use the progress flag when pinning

ipfs pin add --progress <hash>

to see if it stalls for too long.

So if you know you have most of the files locally you should be able to just recursively add the directories that have the content, and pin them after the fact, even if pin=false for the first time, it will notice the blocks are children of the hash you want to pin and prevent them from being considered garbage.

For what it's worth I use badger, I hear large directories can be slow with flatfs depending on your real file system. badger is much better about this but also takes some more resources.

It's also experimental

https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#badger-datastore

Is this dead? Every time I come here to check for new things it hasn't been updated in weeks and I always am the one to necrobump it. Just me. Alone. *sigh*

>>890756

no, i am here

>>890756

Sorry /tech/s post rate isn't fast enough to satiate your ADHD.

They just pushed a release, someone posted about it, multiple people are responding to it, some people are asking questions. What more do you want?

>>891059

I should clarify this by saying you add with "ipfs add --local" and then once the final hash is printed out, put it in "ipfs dht provide [hash]" Again, not really sure if that fixes things anymore so I'd roughly benchmark it before putting it to work.

gm.dls but in .sf2 format

/ipfs/QmeaYGETunJMxtnXQFZhh52YNMYb26qDhDUjVn9YficoST

are there any ipfsites like cool-rom on ipfs? i was thinking of making one

>>894430

Not that I know of. That would be cool to see and practical to mirror, even in parts.

I think most people in the /v/ share threads just post links to folder with the files. There's no frontend, catalog, search, etc.

There's these 2 things that you might want to look at.

Basic anime tracker concept with javascript search

/ipfs/QmXta6nNEcJtRhKEFC4vTB3r72CFAgnJwtcUBU5GJ1TbfU

Atari JS emulator with rom selection, all in browser/on ipfs.

/ipfs/QmacAqRVhJX9eS7YJX1vY3ifFKF9CduDqPEgaCUSa4x5xb/

I think having some way of hosting it on IPFS but having dynamic search clientside like the anime tracker would be interesting. There's a lot of ways you could do that. Generating and searching a static index, using orbit-db, or even some other method.

Same with embedding JavaScript emulators on the page that just pull the rom from IPFS.

If you end up experimenting, post about it here. I'm interested.

>>894490

ill try working on such an idea today

>>894490

I've been thinking about making a IPFS hash + torrent index for a while. Distributed websites are something like the examples you posted where every client is also hosts the entire database. While this would generally work for us on a small scale, as the database grows to hundreds of megabytes or gigabytes it becomes a problem. New users would have to wait to download the entire database, or at least the db index, before they can search for items. This poses scaling and performance issues, though tolerable ones considering the circumstances. The website would also be distributed on IPFS. It would just be a single page with javascript to query the databases.

I'm not sure how best to handle adding new content to the site though. Right now people have been sending information out-of-band through email for the site owner to manually add new content. We could have a bot watch an email/IRC for users to submit content and automatically publish it on the site on the owner's behalf. A report system would also work similarly by notifying the owner of user flagged content to possibly remove. Submit and report forms would be integrated into the website.

>>896606

>New users would have to wait to download the entire database

I don't think that's a hard requirement, there's probably a sane way of sharding the data and hashing queries beforehand that would help here. Maybe OrbitDB already does this.

As for replication I'd expect ipfs-cluster to be useful, or even just having segments users can mirror like how LibGen does it with their torrents.

Either way, my copy of Nyaa is less than 300MB, so even if you made a popular tracker you'd have some time to figure out the backend before it's too large to handle.

>I'm not sure how best to handle adding new content to the site though.

For public dynamic content, look into pubsub on IPFS.

The owner can subscribe to a topic and users can publish to it.

Alternatively, if you'd rather information be sent directly to your node privately, you can have ipfs setup a TCP relay for you. (ipfs p2p --help)

https://github.com/ipfs/go-ipfs/issues/3994

For that you just give out your node PeerID, clients dial it, send data, and you process it however you want. You handle it like a regular TCP connection while IPFS does all the P2P work itself.

>>896894

>I'm not being sarcastic, I like your graphics.

Thanks, I find them to be helpful summaries.

>I wonder how Filecoin is coming along

Considering they raised over $250 million I'm betting they didn't even start yet. IPFS has to become much more mature before they can build on top of it. I remember them saying it's going to be a few years until it's ready.

Private trackers might want to turn on coins on their blockchain to keep track and enforce user upload/download stats in a decentralized way. Public trackers on the other hand wouldn't have much use of coins except maybe as info stats like how BakaBT used to keep track of user upload/download even though it didn't mean anything. I'm betting most public trackers won't use them which is fine.

I want this system to leave as much freedom as possible to the individual websites. I want it to be as easy to add to existing websites as possible. I think of it as a federation of independent islands. There would be a few shared passive rules but the rest is up to the website owner on how to run their own site.

>how to manage distributed moderation

The imageboard thread is >>785171 I posted there previously too. While I agree the user contributed whitelist/blacklist is a good idea for moderating imageboards, I don't think there's much of a moderation issue on torrent sites. Website owners should retain the ability to be able to moderate however they want, otherwise they won't join the system. That said, local copies of the website DBs would have the whole revision history available. Sure someone could write a script to revert specific changed entries but I honestly don't think many people would be doing that. Torrent site moderation and imageboard moderation in practice are really different.

>you could subscribe to an autist who makes sure to filter out anything that isn't 10bit flac encodes

There's no reason why that's not possible with the current setup. I could autistically go through every lossless audio torrent on Nyaa.si and post a whitelist of torrent IDs that were 10bit or higher right now. I could make a RSS feed so others could subscribe and automatically download all 10+bit lossless audio that I approve of.

>I wonder if something like this could work for distributed sites

That's how I would do it for open distributed systems like what I was talking about before using a DHT database for an IPFS website. In fact because of the world writable distributed nature there's really no need to blacklist items unless you want to filter out known spam from queries to save some time.

IPFS Companion 2.2.0 brings window.ipfs to your Browser

https://ipfs.io/blog/35-ipfs-companion-2-2-0/

Your node is exposed as window.ipfs on every webpage. The API aims to be compatible with interface-ipfs-core, which means websites can now detect if the property exists in window context and opt-in to use it instead of creating their own js-ipfs node. It saves system resources and battery (on mobile), avoids the overhead of peer discovery/connection, enables shared repository access and more!

>>897471

doesn't seem to have much users yet, as I only get like 8 nodes connected at a time

Total noob here. I downloaded the go-ipfs thing from their website and tried to ls some of the links ITT, but it says "merkledag: not found" for all of them. Am I missing something?

The example readme and about files do work.

>>897669

that usually means that all the peers hosting the file are offline, i had that problem too

considering that these links were posted on 8chan, they wont have many peers

>>897680

Turned out I had forgotten to start the daemon.

But then I tried it again, and it just stopped right there, seemingly without downloading anything. I can't even do a 'ls'.

>>897684

In fact, this is getting extremely frustrating. Is there a verbose mode? I can't find it from the help.

>>897684

>>897694

you're basically getting the same type of error as trying to download a torrent with no peers, but i would suggest asking the ipfs channel on freenode

>>897719

Alright. Can you confirm you can list /ipfs/QmcdiWeCUHXMEvprg47GhQ6pLXVgpkXDNUZUVnCJG8PfvV ?

Should be a folder with a Makefile, an executable and a source.

>>897723

yep, it shows up in the browser at least

Animegataris 1080p, episodes 1 to 12: /ipfs/QmaYKNfeRc7a2YGTn2qxg9Updn357ouQBX5UL35RxXPN8T

Shoujo Shuumatsu Ryokou 1080p, episodes 1 to 12: /ipfs/QmchtNakPsVoZ7RtZ2siGrK6V8uMkELjt8pfDxMiQ2CYan

Death March kara Hajimaru Isekai Kyousoukyoku 1080p, episodes 1 to 12: /ipfs/QmQMkTQFkjXdF8ZTRLWTjE8QjDiSTJ1KYVkvjWrjKB2GRr

Has someone made a system similar to nyaa for people to list their stuff? Right now it's kind of disorganized with random shit being listed on ephemeral threads and websites. If someone hasn't created such a website yet, and if there's a demand, I think I will do it. What do you guys think?

>>897816

5 cm/s 1080p: /ipfs/QmeWkk3YPdHTpMCy4mVVF8V6EVopoBd44T9GVPMrojAT3m

SAO: Ordinal Scale 1080p: /ipfs/QmPx4TXVqMsmfgtyHPm3XTp92U46dQTsjyFEtzuYtz9kXv

>>897829

We've been discussing ideas to try and figure out what would actually take off. In my opinion a Nyaa clone with IPFS hashes isn't moving anything forward. It just trades torrents for IPFS which is useless when bigger and better alternatives exist. If you just want to have fun go right ahead, but I think we should focus on planing a real successor to the current systems.

>>897953

>trades torrents for IPFS which is useless

Why do you feel that way? IPFS has some innate advantages over bittorrent and the the flexibility leaves room for enhancments.

As it's been said in these threads before, the biggest advantage is the content having a standard address per piece of content, so the swarm per file is always going to be global, unlike torrents where the swarm is grouped by the torrent itself. If we both make a torrent with 2 files, and 1 of those files is the same, people will still only download from one of us, not both. With IPFS it doesn't matter, you'll get the content from whoever will provide it.

This alone is great for longevity. I hate having to download separate torrents for the same group of files and piece a release together just because the seed base has fragmented. Even though 100% of the content is available on the DHT, we don't even request it with bittorrent.

>>898179

>Why do you feel that way?

I should've elaborated. Switching to IPFS is pretty much useless in its current state. That will change when some experimental features mature and become defaults.

>content having a standard address per piece of content

I agree with the design advantages. In fact I've explained that talking point a few times to people in these threads. The problem is that by default it's currently not enabled. The vast majority of users aren't taking advantage of it. Also based on what I've read it only works right now if the hash algorithms match. Say someone enables sharding and adds their 4TB anime folder with the default SHA256 hash. If I do the same but instead use the faster Blake2b hash, my shards and his shards won't be compatible. This will further fragments things since the devs said they're going to change the default to Blake2b in an upcoming update.

I do believe IPFS is a great successor to Bittorrent but it's not quite there yet. What I want to do is discuss, plan and possibly develop the next generation file sharing platform that uses IPFS and open database indexes as opposed to continuing the current closed off centralized torrent sites with IPFS tacked on.

>>898326

>in its current state

Ahh, fair enough. I expect them to nail all this down in the alpha and the defaults not changing much post-release. Glad to see them actually experimenting while they have the chance. I was pretty content with how things were before and they've only gotten better, when they finally group all the best improvements together into a default set, it's going to be nice to see. The concepts are there and work, but seeing them work more efficiently is always a plus.

>What I want to do is discuss, plan and possibly develop the next generation file sharing platform that uses IPFS and open database indexes as opposed to continuing the current closed off centralized torrent sites with IPFS tacked on.

Right on. IPFS or otherwise I just want a less centralized, more efficient platform than what we have now. It's embarrassing we've let this go on for so long despite knowing and regularly facing all the shortcomings of traditional systems. We have all this technology and we utilize it so poorly.

>>897862

Killing Bites, episodes 1 to 12: /ipfs/QmdPjrEMH8eVJoGVp79RkyesUQ3LuBhukp1AF3MEmU1Au9

>>898931

Yuru Camp, episodes 1 to 12: /ipfs/zDMZof1kpFnRGBsgDcBLgAqeu1CjeeGEm9foz3eW9ri96GtX3r8M

Isekai Shokudou, episodes 1 to 12: /ipfs/zDMZof1m1DuN8opbcVhoGc1P3izhJpv87wo8FJAZiH2FY1xafUt1

After the Rain, episodes 1 to 12: /ipfs/zDMZof1m33UyeU3P8pnr3E7yJkKnteQGX63zHxmcPqxyWtNwqHUC

Does anyone know how I can use blake2b-256 hashing algo by default without having to specify on the command line every time?

>>899130

>Does anyone know how I can use blake2b-256 hashing algo by default without having to specify on the command line every time?

https://en.wikipedia.org/wiki/Alias_(command)

Also see the first half of this >>883736

>>899229

I know I could do it on a shell level. I meant whether I could set any ipfs config files to achieve that effect. e.g. if invoking ipfs from a script.

>>899341

>set any ipfs config files to achieve that effect

I don't think so. Maybe worth posting on the issue tracker.

>invoking ipfs from a script

Use variables for the args.

ipfs $IPFS_ARGS

>no anonymity

>can't do shit on it

>won't even let you use tor

==OH NO NO NO NO HAAHAHAHAHAHAHAHA=

>>901143

>fails at redtexting

OH NO NO NO NO HAAHAHAHAHAHAHAHA

>>901143

>won't even let you use tor

What do you mean? You can tunnel it just like normal and there's the fork that has the experimental libp2p-onion transport.

See the second half of >>856739

You've been able to use Tor directly since 2016.

Why don't people read the thread first.

>>793208 (OP)

Can anyone explain me whats the difference between this and dapps on ether? because those are more expensive than regular servers and users have to pay just to access the network to use them so the whole decentralization thing is fucking pointless.

>>901364

They're very different, with IPFS you or someone else stores content, on "blockchains"(merkle trees with root consensus) like Etherium, the entire network distributes the storage essentially.

With IPFS, you hold the data that you hold, with blockchains you either store the entire thing or the tail of it, but it includes the entire history of events.

IPFS is like a more flexible bittorrent, Etherium is more like a distributed computer.

A lot of people have been using them together to have "serverless" Dapps, they use IPFS for anything static and they use Etherium for dynamic stuff. Someone hosts the content on an IPFS instance but the entire etherium network takes the dynamic data, processes it, and stores the result for the entire world to see. It's considered serverless because the network as a whole is storing state, not a single machine. Likewise with IPFS someone is hosting the content, but it's not a typical centralized HTTP server, it could be 1 or many nodes located anywhere over the world and transported through many different ways.

I don't know if this is appropriate but the idea of "what if bittorrent and git had a baby and that baby could be used in a web browser easily", that's an oversimplification of IPFS but maybe works.

Now though IPFS is getting some really good dynamic mechanisms. Read a few posts up about making a service via a p2p socket or via pubsub. It's real time, ephemeral (unless you store it obviously), dynamic, p2p data. The blockchain is more like dynamic, unknown-time, permanent, always public, data. While blockchains are distributed via p2p mechanisms it's a bit odd because it still feels centralized in that way, it's only distributed insofar that you can do things offline but they must be synced up with everyone at some point.

These 2 things are hard to compare, they really do serve different purposes and compliment each other well.

>>901172

https://github.com/ipfs/notes/issues/37

idk wtf you are talking about anon. this issue has been open for 3 years and its still open. tor support BTFO

>>901443

Consider reading the things you link before letting everyone know you're an idiot. The TOR transport is already up and running and you can use it today, but they're waiting on a security audit before putting their official stamp of approval on it.

>>901457

Right so the thing that has been in dev for years is now for real right around the corner. I bet you think the next version of GIMP is gonna come out in 6 months too.

>>901457

IPFS always makes it sound like things are almost done. They have played this game for several fucking years with so many features. Basic things still missing.

>>901465

>>901466

So it begins, the BTFOing of the IPFS meme by people who actually live in the real world.

>>901465

>>901466

>expecting cuckchanners who shill this trash to know anything but their happy shiny fantasy world

Fair warning, you should look out for shit threads like this for possible blatant facts denial. That thid thread got 300+ replies and is only now getting redpilled is shocking.

>>901465

>right around the corner

You're still not listening, it's out right now. You can run it through TOR today, you don't need to wait.

>>901514

Well lets see, from the latest developer comment on the issue page you linked from 20 days ago:

>However, you shouldn't expect a blessed and fully functional/recommended tor transport for a while.

From another one:

>But... there's no documentation. On the other hand, go-onion-transport is at least well-maintained. On the gripping hand, lgierth of ipfs/infrastructure fame admits that:

What we have here are some officially unsupported, and undocumented features.

Now lets look at this actual transport that they implemented

https://github.com/OpenBazaar/go-onion-transport

Oh look it was last edited 7 months ago with absolutely no active work for the past half a year.

>>901143

>>901443

>>901472

>>901476

>>901539

Looks like salty ZeroNet shills came to ruin the discussion. Stay mad faggot. Reminder that IPFS raised $250 million from their ICO.

>>901465

>I bet you think the next version of GIMP is gonna come out in 6 months too

Funny you say that, it shows just how ignorant you are. 2.10 RC2 came out this week with only 7 blocker bugs away from its release. So yes, I do think the next version is gonna come out within 6 months. It could very well come out in 6 weeks.

The Hitler's War audiobook: /ipfs/zDMZof1m3sno11X7CSaJoiWFGabfsuK3HoTS7gVqES57tYyB6YsH

Apocalypse 1945 - Destruction of Dresden: /ipfs/zDMZof1m4ZpRxU7spWcLkScxj3U6WU6vxFMoWNpECw1cWtcsaZjc

>>901547

>b-b-b-b-b-but zeronet!!!!!!

Just as much of a meme as IPFSoy

>Reminder that IPFS raised $250 million from their ICO.

>250 million flies can't be wrong

>>901552

>i don't like it

>no coherent arguments

>>901547

Why are you saging the thread

>>901547

>It could very well come out in 6 weeks.

>6 weeks

LOL try 6 years