Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve ipfs util #440

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions template/web/next.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,18 @@ const nextConfig = withPWA({
protocol: 'https',
hostname: 'ipfs.io',
},
{
protocol: 'https',
hostname: 'cloudflare-ipfs.com',
},
{
protocol: 'https',
hostname: 'gateway.pinata.cloud',
},
{
protocol: 'https',
hostname: 'nftstorage.link',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that's dope. Teach me a bit, how do they work together?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mostly telling nextjs to allow to load images from these domains in nextjs NextImage component

},
],
},
});
Expand Down
1 change: 1 addition & 0 deletions template/web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"clsx": "^2.1.0",
"graphql": "14",
"graphql-request": "6",
"is-ipfs": "6.0.2",
"net": "^1.0.2",
"next": "13.5.6",
"next-pwa": "5.6.0",
Expand Down
132 changes: 121 additions & 11 deletions template/web/src/utils/ipfs.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,128 @@
import { cid } from 'is-ipfs';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General thinking, I like a lot what's going on here.

Why some of these utility don't live in is-ipfs or, do we see other libraries have some of those?

Or should any of those live inside OnchainKit?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Yuripetusko thoughts on this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is-ipfs is great, but it mostly just checks if url has ipfs, it doesn't have utils to replace gateway or add gateway and preserve the path etc. But regarding onchainkit I agree, I can create a similar PR there if this makes sense.

I actually have all of this code in a library already. Just need to add docs. Perhaps instead of having is-ipfs and all this code explicitly, our library will work? Otherwise I can just duplicate some of it in Onchainkit

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's probably move part of it inside OnchainKit.

We can proabably create a new section called IPFS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok will do tomorrow. I'll close this PR then and later will create a new one where onchainkit utils are used


export const DEFAULT_IPFS_GATEWAY_KEYS = {
cloudflare: 'cloudflare',
ipfsIo: 'ipfsIo',
pinata: 'pinata',
nftStorage: 'nftStorage',
};

export type DefaultIpfsGatewayKeys =
(typeof DEFAULT_IPFS_GATEWAY_KEYS)[keyof typeof DEFAULT_IPFS_GATEWAY_KEYS];

export type IpfsGateways = Record<DefaultIpfsGatewayKeys, string>;

export const DEFAULT_IPFS_GATEWAY_HOSTNAMES: IpfsGateways = {
cloudflare: 'cloudflare-ipfs.com',
ipfsIo: 'ipfs.io',
pinata: 'gateway.pinata.cloud',
nftStorage: 'nftstorage.link',
};

/**
* Checks if a string contains an IPFS CID
* @param ipfsURI
* @returns an object with a boolean indicating if the string contains a CID and the CID if it does
*/
export const containsCID = (ipfsURI?: string | null) => {
if (typeof ipfsURI !== 'string') {
throw new Error('url is not string');
}
const splitUrl = ipfsURI.split(/\/|\?/);
for (const split of splitUrl) {
if (cid(split)) {
return {
containsCid: true,
cid: split,
};
}
const splitOnDot = split.split('.')[0];
if (cid(splitOnDot)) {
return {
containsCid: true,
cid: splitOnDot,
};
}
}

return {
containsCid: false,
cid: '',
};
};

/**
* Transforms an IPFS URL to a desired gateway.
* The input URL can either already be a http url or an ipfs uri.
* Supports ipns and ipfs uris with folder paths
* @param ipfsURI - ipfs uri or http url (ipfs://${cid} or http://{gatewayHostname | 'ipfs.io'}/ipfs/${cid})
* @param gatewayHostname - preferred gateway provider
*/
export const transformIpfsUrlToUrlGateway = (
ipfsURI?: string | null,
gatewayHostname = DEFAULT_IPFS_GATEWAY_HOSTNAMES[DEFAULT_IPFS_GATEWAY_KEYS.ipfsIo],
) => {
const results = containsCID(ipfsURI);
if (!ipfsURI || !results.containsCid || !results.cid) {
throw new Error('url does not contain CID');
}

if (gatewayHostname?.startsWith('http')) {
throw new Error('gatewayHostname should not start with http');
}

//Get url parts after the cid, to append them back to the new gateway url
const splitUrl = ipfsURI.split(results.cid);
//case 1 - the ipfs://cid path
if (ipfsURI.includes(`ipfs://${results.cid}`)) {
return `https://${gatewayHostname}/ipfs/${results.cid}${splitUrl[1]}`;
}

//case 2 - the /ipfs/cid path (this should cover ipfs://ipfs/cid as well
if (ipfsURI.includes(`/ipfs/${results.cid}`)) {
return `https://${gatewayHostname}/ipfs/${results.cid}${splitUrl[1]}`;
}

//case 3 - the /ipns/cid path
if (ipfsURI.includes(`/ipns/${results.cid}`)) {
return `https://${gatewayHostname}/ipns/${results.cid}${splitUrl[1]}`;
}

if (!ipfsURI.includes('ipfs') && ipfsURI === results.cid) {
return `https://${gatewayHostname}/ipfs/${results.cid}${splitUrl[1]}`;
}

console.warn(`Unsupported URL pattern: ${ipfsURI}. Attempting a default fallback.`);
return `https://${gatewayHostname}/ipfs/${results.cid}${splitUrl[1]}`;
};

/**
* Convert IPFS URI to HTTPS URI.
* Convert IPFS or HTTP URI to HTTPS URI with an ipfs gateway provider (if provided).
*
* @param ipfsURI An ipfs protocol URI.
* @param gateway The IPFS gateway to use. Defaults to ipfs.io, a free public gateway.
* @param ipfsURI A value from contract. Can be an ipfs or http URI.
* @param gatewayHostname The IPFS gateway to use. Defaults to ipfs.io, a free public gateway.
* For production use, you'll likely want a paid provider.
* @returns An HTTPS URI that points to the data represented by the cid
* embedded in the ipfs URI.
*/
export const ipfsToHTTP = function (ipfsURI: string, gateway = 'ipfs.io') {
// IPNS Name is a Multihash of a serialized PublicKey.
const cid = ipfsURI.replace('ipfs://', '');

// Addresses using a gateway use the following form,
// where <gateway> is the gateway address,
// and <CID> is the content identifier.
return `https://${gateway}/ipfs/${cid}`;
export const ipfsToHTTP = (ipfsURI?: string | null, gatewayHostname?: string) => {
if (!ipfsURI) return '';

if (ipfsURI.startsWith('http') && !containsCID(ipfsURI).containsCid) {
return ipfsURI.replace('http://', 'https://');
}

// If no gateway url is passed, and url contains cid, and url already starts with http, just return it as is, making sure it's https
if (ipfsURI.startsWith('http') && containsCID(ipfsURI).containsCid && !gatewayHostname) {
return ipfsURI.replace('http://', 'https://');
}

if (containsCID(ipfsURI).containsCid) {
return transformIpfsUrlToUrlGateway(
ipfsURI,
gatewayHostname ?? DEFAULT_IPFS_GATEWAY_HOSTNAMES[DEFAULT_IPFS_GATEWAY_KEYS.ipfsIo],
);
}

return '';
};
20 changes: 20 additions & 0 deletions template/web/src/utils/test/ipfsToHTTP.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,24 @@ describe('ipfsToHTTP', () => {
const expected = 'https://ipfs.io/ipfs/QmY5V6JZ5Yf7Z6p8n7Y1Z5dJLXhZU7Z7Q3mH2nX8vqfHc5';
expect(ipfsToHTTP(ipfsURI)).toEqual(expected);
});

it('converts ipfs URI with a folder path to HTTPS URI', () => {
const ipfsURI = 'ipfs://QmT7iXqGBP3RiWrAcM6PQwtyqeXPsHpUDLatpNo8gNFDfK/metadata/1.json';
const expected =
'https://ipfs.io/ipfs/QmT7iXqGBP3RiWrAcM6PQwtyqeXPsHpUDLatpNo8gNFDfK/metadata/1.json';
expect(ipfsToHTTP(ipfsURI)).toEqual(expected);
});

it('converts ipfs http URL to https url', () => {
const ipfsURI = 'http://ipfs.io/ipfs/QmY5V6JZ5Yf7Z6p8n7Y1Z5dJLXhZU7Z7Q3mH2nX8vqfHc5';
const expected = 'https://ipfs.io/ipfs/QmY5V6JZ5Yf7Z6p8n7Y1Z5dJLXhZU7Z7Q3mH2nX8vqfHc5';
expect(ipfsToHTTP(ipfsURI)).toEqual(expected);
});

it('converts ipfs URL to HTTPS URL with provided gateway', () => {
const ipfsURI = 'http://ipfs.io/ipfs/QmY5V6JZ5Yf7Z6p8n7Y1Z5dJLXhZU7Z7Q3mH2nX8vqfHc5';
const expected =
'https://cloudflare-ipfs.com/ipfs/QmY5V6JZ5Yf7Z6p8n7Y1Z5dJLXhZU7Z7Q3mH2nX8vqfHc5';
expect(ipfsToHTTP(ipfsURI, 'cloudflare-ipfs.com')).toEqual(expected);
});
});
112 changes: 111 additions & 1 deletion template/web/yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -1560,6 +1560,7 @@ __metadata:
graphql: "npm:14"
graphql-request: "npm:6"
identity-obj-proxy: "npm:^3.0.0"
is-ipfs: "npm:6.0.2"
jest: "npm:^29.7.0"
jest-environment-jsdom: "npm:^29.7.0"
jest-extended: "npm:^4.0.2"
Expand Down Expand Up @@ -7546,6 +7547,17 @@ __metadata:
languageName: node
linkType: hard

"dns-over-http-resolver@npm:^1.2.3":
version: 1.2.3
resolution: "dns-over-http-resolver@npm:1.2.3"
dependencies:
debug: "npm:^4.3.1"
native-fetch: "npm:^3.0.0"
receptacle: "npm:^1.3.2"
checksum: 10c0/231435742246115aeb4f153721effc4d995ab8f22572240b27d85e1be4123345cbe503e9922bc46b36caaa86307fbcf65ba252302dc7a4794f330aa6d6f920b8
languageName: node
linkType: hard

"doctrine@npm:^2.1.0":
version: 2.1.0
resolution: "doctrine@npm:2.1.0"
Expand Down Expand Up @@ -7761,6 +7773,13 @@ __metadata:
languageName: node
linkType: hard

"err-code@npm:^3.0.1":
version: 3.0.1
resolution: "err-code@npm:3.0.1"
checksum: 10c0/78b1c50500adebde6699b8d27b8ce4728c132dcaad75b5d18ba44f6ccb28769d1fff8368ae1164be4559dac8b95d4e26bb15b480ba9999e0cd0f0c64beaf1b24
languageName: node
linkType: hard

"error-ex@npm:^1.3.1":
version: 1.3.2
resolution: "error-ex@npm:1.3.2"
Expand Down Expand Up @@ -9916,6 +9935,20 @@ __metadata:
checksum: 10c0/331cd07fafcb3b24100613e4b53e1a2b4feab11e671e655d46dc09ee233da5011284d09ca40c4ecbdfe1d0004f462958675c224a804259f2f78d2465a87824bc
languageName: node
linkType: hard

"ip-regex@npm:^4.0.0":
version: 4.3.0
resolution: "ip-regex@npm:4.3.0"
checksum: 10c0/f9ef1f5d0df05b9133a882974e572ae525ccd205260cb103dae337f1fc7451ed783391acc6ad688e56dd2598f769e8e72ecbb650ec34763396af822a91768562
languageName: node
linkType: hard

"ip@npm:^2.0.0":
version: 2.0.0
resolution: "ip@npm:2.0.0"
checksum: 10c0/8d186cc5585f57372847ae29b6eba258c68862055e18a75cc4933327232cb5c107f89800ce29715d542eef2c254fbb68b382e780a7414f9ee7caf60b7a473958
languageName: node
linkType: hard

"iron-webcrypto@npm:^1.0.0":
version: 1.0.0
Expand Down Expand Up @@ -10103,6 +10136,28 @@ __metadata:
checksum: 10c0/a8efb0e84f6197e6ff5c64c52890fa9acb49b7b74fed4da7c95383965da6f0fa592b4dbd5e38a79f87fc108196937acdbcd758fcefc9b140e479b39ce1fcd1cd
languageName: node
linkType: hard

"is-ip@npm:^3.1.0":
version: 3.1.0
resolution: "is-ip@npm:3.1.0"
dependencies:
ip-regex: "npm:^4.0.0"
checksum: 10c0/4cb643c831314b8fc72770c93a795c0d3dde339f36c8430544c36727956027e2cb329641ace73c5951085ecf93ac608c898859d3d4f7b117d405e1e13c703c76
languageName: node
linkType: hard

"is-ipfs@npm:6.0.2":
version: 6.0.2
resolution: "is-ipfs@npm:6.0.2"
dependencies:
iso-url: "npm:^1.1.3"
mafmt: "npm:^10.0.0"
multiaddr: "npm:^10.0.0"
multiformats: "npm:^9.0.0"
uint8arrays: "npm:^3.0.0"
checksum: 10c0/4f2e91e4a8fefba2254ff650ec7f3f5386475b5440708cd04e29a911133732cf4697c2ab968c3f9cdfe7a52bff341652e36acbf563bb65f45feead44a5078e8c
languageName: node
linkType: hard

"is-lambda@npm:^1.0.1":
version: 1.0.1
Expand Down Expand Up @@ -10377,6 +10432,13 @@ __metadata:
languageName: node
linkType: hard

"iso-url@npm:^1.1.3":
version: 1.2.1
resolution: "iso-url@npm:1.2.1"
checksum: 10c0/73be82eaaf5530acb1b6a46829e0dfb050c62790b8dc04d7fb7e290b63c88846b4d861ecf3a6bc7e0a3d74e569ea53c0fb951d596e06d6c6dd0cf4342d59ecc9
languageName: node
linkType: hard

"isomorphic-unfetch@npm:3.1.0":
version: 3.1.0
resolution: "isomorphic-unfetch@npm:3.1.0"
Expand Down Expand Up @@ -11600,6 +11662,15 @@ __metadata:
languageName: node
linkType: hard

"mafmt@npm:^10.0.0":
version: 10.0.0
resolution: "mafmt@npm:10.0.0"
dependencies:
multiaddr: "npm:^10.0.0"
checksum: 10c0/6f5e0aa964326518f44e19d57db73fb7b7d88ca9d29ea42371c76029b0cfe381d522fcd558a3fd8364868b54fb7f6fa012f730fb7270bce00baa9a866d428664
languageName: node
linkType: hard

"magic-string@npm:^0.25.0, magic-string@npm:^0.25.7":
version: 0.25.9
resolution: "magic-string@npm:0.25.9"
Expand Down Expand Up @@ -12294,7 +12365,21 @@ __metadata:
languageName: node
linkType: hard

"multiformats@npm:^9.4.2":
"multiaddr@npm:^10.0.0":
version: 10.0.1
resolution: "multiaddr@npm:10.0.1"
dependencies:
dns-over-http-resolver: "npm:^1.2.3"
err-code: "npm:^3.0.1"
is-ip: "npm:^3.1.0"
multiformats: "npm:^9.4.5"
uint8arrays: "npm:^3.0.0"
varint: "npm:^6.0.0"
checksum: 10c0/948d0c69d75992d754fd36154db4d4a435b977c56a617183a9d8c2075081e2cfcacda3bedb72bbd6759bb329956a45b216e804829e708af8d2f780f8152a4bf6
languageName: node
linkType: hard

"multiformats@npm:^9.0.0, multiformats@npm:^9.4.2, multiformats@npm:^9.4.5":
version: 9.9.0
resolution: "multiformats@npm:9.9.0"
checksum: 10c0/1fdb34fd2fb085142665e8bd402570659b50a5fae5994027e1df3add9e1ce1283ed1e0c2584a5c63ac0a58e871b8ee9665c4a99ca36ce71032617449d48aa975
Expand Down Expand Up @@ -12347,6 +12432,15 @@ __metadata:
languageName: node
linkType: hard

"native-fetch@npm:^3.0.0":
version: 3.0.0
resolution: "native-fetch@npm:3.0.0"
peerDependencies:
node-fetch: "*"
checksum: 10c0/737cdd209dd366df8b748dabac39340089d57a2bcc460ffc029ec145f30aeffea0c6a6f177013069d6f7f04ffc8c3e39cfb8e3825e7071a373c4f86b187ae1b5
languageName: node
linkType: hard

"natural-compare-lite@npm:^1.4.0":
version: 1.4.0
resolution: "natural-compare-lite@npm:1.4.0"
Expand Down Expand Up @@ -13926,6 +14020,15 @@ __metadata:
languageName: node
linkType: hard

"receptacle@npm:^1.3.2":
version: 1.3.2
resolution: "receptacle@npm:1.3.2"
dependencies:
ms: "npm:^2.1.1"
checksum: 10c0/213dc9e4e80969cde60c5877fae08d8438f0bf7dd10bf4ea47916a10c053ca05d6581bda374d8f22ce15e6b50739efe319d847362f5ec9e1a4cbdcbde3ddf355
languageName: node
linkType: hard

"redent@npm:^3.0.0":
version: 3.0.0
resolution: "redent@npm:3.0.0"
Expand Down Expand Up @@ -16355,6 +16458,13 @@ __metadata:
languageName: node
linkType: hard

"varint@npm:^6.0.0":
version: 6.0.0
resolution: "varint@npm:6.0.0"
checksum: 10c0/737fc37088a62ed3bd21466e318d21ca7ac4991d0f25546f518f017703be4ed0f9df1c5559f1dd533dddba4435a1b758fd9230e4772c1a930ef72b42f5c750fd
languageName: node
linkType: hard

"vfile-location@npm:^5.0.0":
version: 5.0.2
resolution: "vfile-location@npm:5.0.2"
Expand Down
Loading