Suck a DAG out of a peer in the IPFS network.

Overview

dagula

Build JavaScript Style Guide

Suck a DAG out of a peer in the IPFS network.

Install

npm i dagula

Usage

import { Dagula } from 'dagula'
import { getLibp2p } from 'dagula/p2p.js'
import { CID } from 'multiformats/cid'

const libp2p = await getLibp2p()
const peer = '/dns4/peer.ipfs-elastic-provider-aws.com/tcp/3000/ws/p2p/bafzbeibhqavlasjc7dvbiopygwncnrtvjd2xmryk5laib7zyjor6kf3avm'
const dagula = new Dagula(libp2p, peer)

// fetch entire DAG
const cid = 'bafybeig4qjehigdddcoka23crh2s3vrautbep3topuoqblb4chkvvhpilu'
for await (const block of dagula.get(cid)) {
  console.log(`${block.cid} (${block.bytes.length} bytes)`)
}

// fetch a file/directory
const path = 'bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm/2998.png'
for await (const entry of dagula.getUnixfs(path)) {
  console.log(`${entry.path} (${entry.size} bytes)`)
  // note: use `entry.content()` to get file data
}

CLI

# fetch a complete DAG (output format is CAR)
dagula get bafybeidtzj4g33h4d76nfyznjfcejyigejrjpqfzm6ydapuhd26asjg5re > output.car

# fetch a UnixFS file
dagula unixfs get bafybeidtzj4g33h4d76nfyznjfcejyigejrjpqfzm6ydapuhd26asjg5re/path/to/data.txt

# fetch a specific block
dagula block get bafybeidtzj4g33h4d76nfyznjfcejyigejrjpqfzm6ydapuhd26asjg5re

# configure peer to use
dagula peer set /dns4/peer.ipfs-elastic-provider-aws.com/tcp/3000/ws/p2p/bafzbeibhqavlasjc7dvbiopygwncnrtvjd2xmryk5laib7zyjor6kf3avm

Contributing

Feel free to join in. All welcome. Open an issue!

License

Dual-licensed under MIT + Apache 2.0

Comments
  • feat: add `ls` and `tree` commands

    feat: add `ls` and `tree` commands

    • dagula ls – I wanted a quick way to check if a remote node has an entire DAG without needing to save the block data locally (see #2)
    • dagula tree - You wanted a pretty way to see a DAG

    So why not both!? This PR takes the liberty of yeilding the decoded Block rather than the { cid, bytes } combo, so it can access the links info. It's neat because a decoded Block also as cid, and bytes, so it work! 🎉

    On a snapshot of the dagula source we get:

    ls

    ❯ ./bin.js ls bafybeianow3xqswlzqmmqp4enj4mul3snpg52rya4rvpanip5tycw43elm
    bafybeianow3xqswlzqmmqp4enj4mul3snpg52rya4rvpanip5tycw43elm
    bafkreifjt56wbljg574jp3ij3iz56rr3ligwf3tmcwmyqufabuhytiirq4
    bafkreibsrowqy5n4jpr7en25pepmw2psg6mnhoxzm6yodqubw6uqkl3nau
    bafkreid52wbs2t5bjl7k4jqvcmvmpwnwifnvresgdmstcpwhw3h2wb4h6e
    bafkreienidxwfi26e7zz54wgkugorvzuugrkaijievwwbrjal37qsnmvye
    bafkreicaz5es35z2z2rpu63wukkg2ij2vctmsogmviudlbeb4dfpjfya2u
    bafkreicsmlfxu3b4d2ubhlkk5bjo46nnxs5foee7x5bndnn5tpq7ghpo34
    bafybeife4dxf7mfar2qsqgexkhze6b4w43a46rvfeoesnkooc5ktdc4uim
    bafkreieaixgg2evxchw4wwzzcegecjayq2dn7ka7dblps425royn7ujg5m
    bafkreiekuaqztmk7upvdftvq5lwjfmwbg3ffsl7l2oxayrjlqlbj3egcwm
    bafkreihblq7u4qvbte34agifm7cjhaerpwxt5q53xy4jnfw47im5hzlqdi
    bafkreifpiqrfihldtiziyos6u2k35lf45qrewid2hoylaj5uyi35u5xmbq
    bafkreihrp4ceuqkpfl7ytv4kk5qn2ztsyi2weka6gmc346iixlqzupmw5a
    bafkreiencnfom77cudkikff7dyy7osc5e22vqset7vpotcyudv274xilq4
    bafkreihvxng4xlbbpqrd2ob67btegs3pnocsqf6bq7ahqqo7ofyx7payhy
    bafkreic3qg76debp4uwj5hnfpawaqbcv6qyptwn2sz4oxp73md66kdtree
    bafkreia5kv2iq3ewgdravhtrlz7zeywhm6cynwx5xieggjl4dlysv772t4
    bafkreiejwjgkdtnox6pulwpa4gyjyuczbz4t3k7waeq64i5lgbxjdfifri
    

    tree

    ❯ ./bin.js tree bafybeianow3xqswlzqmmqp4enj4mul3snpg52rya4rvpanip5tycw43elm
    bafybeianow3xqswlzqmmqp4enj4mul3snpg52rya4rvpanip5tycw43elm
    ├── bafkreifpiqrfihldtiziyos6u2k35lf45qrewid2hoylaj5uyi35u5xmbq
    ├── bafkreic3qg76debp4uwj5hnfpawaqbcv6qyptwn2sz4oxp73md66kdtree
    ├── bafkreihrp4ceuqkpfl7ytv4kk5qn2ztsyi2weka6gmc346iixlqzupmw5a
    ├── bafkreicaz5es35z2z2rpu63wukkg2ij2vctmsogmviudlbeb4dfpjfya2u
    ├── bafkreihblq7u4qvbte34agifm7cjhaerpwxt5q53xy4jnfw47im5hzlqdi
    ├─┬ bafybeife4dxf7mfar2qsqgexkhze6b4w43a46rvfeoesnkooc5ktdc4uim
    │ ├── bafkreia5kv2iq3ewgdravhtrlz7zeywhm6cynwx5xieggjl4dlysv772t4
    │ └── bafkreiejwjgkdtnox6pulwpa4gyjyuczbz4t3k7waeq64i5lgbxjdfifri
    ├── bafkreieaixgg2evxchw4wwzzcegecjayq2dn7ka7dblps425royn7ujg5m
    ├── bafkreienidxwfi26e7zz54wgkugorvzuugrkaijievwwbrjal37qsnmvye
    ├── bafkreid52wbs2t5bjl7k4jqvcmvmpwnwifnvresgdmstcpwhw3h2wb4h6e
    ├── bafkreifjt56wbljg574jp3ij3iz56rr3ligwf3tmcwmyqufabuhytiirq4
    ├── bafkreicsmlfxu3b4d2ubhlkk5bjo46nnxs5foee7x5bndnn5tpq7ghpo34
    ├── bafkreibsrowqy5n4jpr7en25pepmw2psg6mnhoxzm6yodqubw6uqkl3nau
    ├── bafkreihvxng4xlbbpqrd2ob67btegs3pnocsqf6bq7ahqqo7ofyx7payhy
    ├── bafkreiencnfom77cudkikff7dyy7osc5e22vqset7vpotcyudv274xilq4
    └── bafkreiekuaqztmk7upvdftvq5lwjfmwbg3ffsl7l2oxayrjlqlbj3egcwm
    

    i believe we are seeing the gen dir at bafybeife4dxf7mfar2qsqgexkhze6b4w43a46rvfeoesnkooc5ktdc4uim with 2 children... YES! IT TRUE https://bafybeife4dxf7mfar2qsqgexkhze6b4w43a46rvfeoesnkooc5ktdc4uim.ipfs.w3s.link

    License: MIT Signed-off-by: Oli Evans [email protected]

    opened by olizilla 2
  • Maybe use the same Blockstore interface as @ipld/car

    Maybe use the same Blockstore interface as @ipld/car

    I was noodling around and wondered if I could re-use the dagula decode'n'walk logic on a CAR file. It was a success, but made slightly rougher by the different expectations of what a Block{store|Reader}'s get method signature should be.

    dagular Blockstore return a promise of the bytes or throw

    get: (cid: CID, options?: { signal?: AbortSignal }) => Promise<Uint8Array>
    

    source

    @ipld/car BlockReader return a promise of {cid: CID, bytes: Uint8Array} or undefined

    get(key: CID): Promise<Block | undefined>
    

    source

    opened by olizilla 2
  • feat: add `dagula walk` to print CIDs as we walk the DAG

    feat: add `dagula walk` to print CIDs as we walk the DAG

    I wanted a quick way to check if a remote node has an entire DAG without needing to save the block data locally, so I added dagula walk.

    It could be also be called refs to match what ipfs refs does, but I went with walk.

    License: MIT Signed-off-by: Oli Evans [email protected]

    opened by olizilla 2
  • feat: make dagula work on node 16

    feat: make dagula work on node 16

    node 16 is LTS. signal.throwIfAborted landed in node 17. This PR makes dagular work on node 16 and tests it.

    see: https://nodejs.org/api/globals.html#abortsignalthrowifaborted see: https://github.com/nodejs/node/commit/3f72c72bbb89154bc15fbcef7ccbe3cf0dd017ba

    Also the setup-node@v3 action now supports caching deps, so we drop bahmutov/npm-install@v1. see: https://github.com/actions/setup-node/blob/main/docs/advanced-usage.md#caching-packages-data

    License: MIT Signed-off-by: Oli Evans [email protected]

    opened by olizilla 1
  • feat!: use js-car blockstore interface

    feat!: use js-car blockstore interface

    BREAKING CHANGE: The blockstore interface used by this library has been updated. If passing a blockstore to the constructor it should return undefined if the block is not present in the blockstore. If it is present, it should return a { bytes: Uint8Array, cid: CID } object (a "Block"). Secondly the getBlock method now returns a Block instead of a Uint8Array.

    resolves https://github.com/web3-storage/dagula/issues/3

    opened by alanshaw 0
Owner
Alan Shaw
Bug Author
Alan Shaw
An Opensource Peer-to-peer Social Network with Zero-Knowledge-Proof based authentication.

HexHoot This is an attempt to create an Opensource Peer-to-peer Social Network with Zero-Knowledge-Proof based authentication. The objective is to dem

Zenin Easa Panthakkalakath 6 Dec 28, 2022
Peer-to-peer terminal chat based on DStack

peerchat Peer-to-peer terminal chat based on DStack Recording looks ugly, but in terminal it seems to be fine peerchat Usage Commands Usage $ npm inst

DStack 7 Aug 17, 2022
Quiet Peer-to-Peer Donations

Quiet Peer-to-Peer Donations

Arcade City 7 Jun 6, 2022
A peer-to-peer chat app that is serverless, decentralized, and ephemeral

Chitchatter Logo provided by @ramyashreeshetty Chitchatter is a free (as in both price and freedom) communication tool. It is designed with security a

Jeremy Kahn 714 Dec 19, 2022
🪐 The IPFS gateway for NFT.Storage is not "another gateway", but a caching layer for NFTs that sits on top of existing IPFS public gateways.

nftstorage.link The IPFS gateway for nft.storage is not "another gateway", but a caching layer for NFT’s that sits on top of existing IPFS public gate

NFT.Storage 37 Dec 19, 2022
A tiny, efficient, fuzzy search that doesn't suck

▒ μFuzzy A tiny, efficient, fuzzy search that doesn't suck Introduction This is my fuzzy ?? . There are many like it, but this one is mine. uFuzzy is

Leon Sorokin 1.9k Dec 25, 2022
Simple but Complete & Fast network monitor for your home network

netmon Netmon is an opensource project for protecting and monitoring your home network. Netmon is written to run on a Raspberry PI and is optimized to

Tommaso Ventafridda 9 Jul 6, 2022
TS & JS Library for adaptive precision cursor for the web. Releases will come out soon! Meanwhile, check out the demo site:

Haha, cool cursor go brrrr... Table of Content What is this? Installation & Setup Installation Setup Usage Cursor controls Element settings Known issu

LemonOrange 10 Nov 24, 2022
An on-demand peer tutoring platform by students, for students.

OURFinals An on-demand peer tutoring platform by students, for students. Database The prisma/ folder contains: migrations/: Past database migrations (

Aditya Banerjee 3 Jan 6, 2022
A robust, minimal-server-interaction API for peer routing in the browser

Robust, minimal-server-interaction peer routing in the browser What is this? Membrane takes signalling to the browser, creating living peer networks.

Elijah Bodden 13 Jan 6, 2023
Group together Tailwind CSS modifiers like focus, peer-checked, dark:hover and more with HTML attributes 👩‍🚀

Tailwind CSS Group Classes Group together Tailwind CSS modifiers like focus, peer-checked, dark:hover and more with HTML attributes ??‍?? Using with a

Mark Mead 5 Sep 15, 2022
potsky.eth NTF website hosted on IPFS

potsky NFT Website Introduction This website showcases potsky's digital creations from 90's to now created on Amiga and on Mac OS X. Dev # install dep

Potsky 2 Jan 6, 2022
Dead simple program to upload NFT data to IPFS via nft.storage

NFTP The simplest way to publish files and folders to IPFS, with one command. 100% FREE to upload as much files as you want, powered by nft.storage. N

factoria 35 Dec 11, 2022
Decentralized twitter using Solidity, Ethereum, hardhat, ethers, IPFS, Next.JS, TypeScript, TailwindCSS.

DWITTER: Decentralized Twitter Check out the deployed version of this app at https://dwtr.wajeshubham.in Transactions on Ethereum are slow. Therefore,

Shubham Waje 12 Sep 2, 2022
Yet another library for generating NFT artwork, uploading NFT assets and metadata to IPFS, deploying NFT smart contracts, and minting NFT collections

eznft Yet another library for generating NFT artwork, uploading NFT assets and metadata to IPFS, deploying NFT smart contracts, and minting NFT collec

null 3 Sep 21, 2022
A CLI to upload files to IPFS and interact with them using wbeb3.storage

Storli A CLI to upload files to IPFS and interact with them using web3.storage Storli Usage Commands Usage $ npm install -g storli $ storli COMMAND ru

Anish De 9 Aug 7, 2022
✨ An IRL tokenization platform to turn your hopes, dreams, and desires into fundable NFTs on the Polygon blockchain using Chainlink, IPFS, Moralis, and NFT.Storage.

GoFundYourself Getting funding for your passion project, needs or dream doesn't have to be a nightmare! check out our live demo on Netlify Let's Fundi

Brian H. Hough | brianhuff.eth 7 Dec 6, 2022
Minty is an example of how to mint non-fungible tokens (NFTs) while storing the associated data on IPFS

Minty is an example of how to mint non-fungible tokens (NFTs) while storing the associated data on IPFS. You can also use Minty to pin your data on an IPFS pinning service such as nft.storage and Pinata.

One & Zeros 10 Nov 12, 2022
ChainLook is a decentralized blockchain analytics platform based on TheGraph and IPFS.

ChainLook https://chainlook.xyz ChainLook is a decentralized blockchain analytics platform based on TheGraph and IPFS. You can create beautiful widget

Saleel 11 Nov 21, 2022