GERA2LD.SPACE

Deploying My Blog to IPFS

Recently I have deployed my blog to IPFS, it is amazing to be a part of the distributed world.

What is IPFS

IPFS powers the Distributed Web

IPFS, aka. InterPlanetary File System, hosts files on a distributed web, which means all files are hosted by peers and cannot be changed or taken down.

Each file will have a unique identity called CID, which is based on its content. So the CID remains the same no matter how many times you upload the same file, and it changes when any change is made to our file.

Because the files are retrieved from peers, the bandwidth of our server will be saved significantly. However it depends on how many people are visiting our website with IPFS. The more people are visiting it, the more people will be hosting it, the faster it will be and the more server bandwidth will be saved.

But if no one is hosting it on IPFS, the website will not be accessible at all, so we need a way to serve it on IPFS by ourselves, we will talk about that later.

So hosting on IPFS can help only if many people are visiting and hosting our website using IPFS, exactly how P2P network works.

It doesn't really help for this website with a little audience so it is just a demonstration and an experiment.

IPFS CAR

CAR is short for Content-addressed Archive. It bundles all files into an archive to allow them to be connected with relative paths.

As mentioned above, files on IPFS are immutable with different CIDs based on their contents. But if we build a website, we have multiple files to retrieve, one based on another.

By bundling them into a CAR, we can just build the website as usual and don't need to worry about the CID for each file -- IPFS will handle it.

CAR file has its own CID too. So each time we rebuild the website, the entrance to our website will be the new CID of the CAR file.

How do users know how to visit our new website? Let's move on to IPNS.

IPNS stands for InterPlanetary Name System. It is the DNS for the IPFS world.

DNSLink is the bridge between traditional DNS and IPNS. It uses DNS TXT records to store the mapping from a traditional domain name to an IPFS CID.

Let's take an example, say we have a website with an entrance CID of QmbezGequPwcsWo8UL4wDF6a8hYwM1hmbzYv2mnKkEWaUp.

So we can visit it from a public gateway like this: https://ipfs.io/ipfs/QmbezGequPwcsWo8UL4wDF6a8hYwM1hmbzYv2mnKkEWaUp.

However we don't want users to input a different address every time, so we create a DNS TXT record for it.

Suppose we own a domain gera2ld.space. As per DNSLink documentation, we should create a DNS record with type of TXT, name of _dnslink.gera2ld.space and value of dnslink=/ipfs/QmbezGequPwcsWo8UL4wDF6a8hYwM1hmbzYv2mnKkEWaUp. Once properly resolved, https://ipfs.io/ipns/gera2ld.space will lead us to the website above.

Pinning Files

IPFS stores its data on a distributed web, in other words, our files are stored in other users' computers as long as they keep our files.

Here comes the questions.

  • When will other users have the files?

    When a user visits our website, they will retrieve the files from other users hosting the files.

  • Why would they keep our files?

    According to the docs, IPFS has a fairly aggressive caching mechanism that will keep our files for a short time after fetching them. During that time other users can also fetch the files from them.

  • What will happen if they don't keep our files?

    The files will be accessible as long as there are users hosting the files. However, if no one has the files any more, the files are gone for good.

You may find a chicken and egg situation here. A user can only host the files if they retrieve the files from another user.

Since we own the original files, we upload it to an IPFS node and that node will be the first user hosting the files.

However the files may disappear after a while if no one is asking for it. In order to make it stay we pin the files.

Now we have a new problem, we can only pin files in our own IPFS node, which means we have to keep an IPFS instance in our server. That is quite unacceptable as it will break my automatic deployment flow.

Web3.Storage

Thanks to blockchains and the distributed world, we got new lovely infrastructure like Web3.Storage.

It allows you to upload files to IPFS and store a copy in the Filecoin network. It is free and allows up to 1TB data per account.

Now we get pretty much all we need to build our IPFS version of website automatically and serverlessly.

Making It Automatic

This website on traditional network is built with Gatsby and hosted on Vercel.

It is quite straightforward to generate and serve a static website. Basically every time when a change is made, Vercel will start a deployment, running gatsby build and serving the generated static files.

Now we are going to send the files to IPFS during the deployment.

We need to make sure ipfs-car is installed.

$ npm i ipfs-car

Suppose gatsby build has finished and we get public/ containing all the static files.

Let's create a CAR file for the static files:

$ ipfs-car --wrapWithDirectory false --pack public --output $CAR_FILE

and keep the CID:

$ CID=$(ipfs-car --list-roots $CAR_FILE)

Then update DNSLink by APIs, this step depends on your DNS provider, for instance updating the record on Cloudflare:

$ curl -fsSL -H "Authorization: Bearer $CLOUDFLARE_TOKEN" -H "Content-Type: application/json" -X PATCH "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID" --data "{\"content\":\"dnslink=/ipfs/$CID\"}"

Finally the most important step, to upload and pin our files.

Web3.Storage has an HTTP API to upload CAR file:

$ curl -fsSL -H "Authorization: Bearer $WEB3STORAGE_TOKEN" -H "Content-Type: application/vnd.ipld.car" -H "X-name: $NAME" -X POST --data-binary @$CAR_FILE https://api.web3.storage/car

$NAME is an optional readable name for us to see in the web UI.

After all the steps above succeed, hopefully we will be able to access the website from a public gateway with https://dweb.link/ipns/gera2ld.space.

Experience Improvement

We are almost done, but we still have a few things to improve.

Let's introduce Brave Browser before moving on. Brave Browser is a Chromium based browser which has almost the same browsing experience as Chrome expect for protecting your privacy better. But this is not why it is here.

Brave Browser supports IPFS natively!

In Brave Browser we can open an IPFS file with a URL like ipfs://<CID> or ipns://<DOMAIN_NAME>.

So we can open this website with just ipns://gera2ld.space. Cool, isn't it?

Another thing is, Brave Browser can show an IPFS badge in the address bar indicating that this website has an IPFS version. When clicking that badge, we will be redirected to the IPFS version. But we need to tell the browser that we have an IPFS version.

To achieve this we need to add an HTTP header in the response of our website: x-ipfs-path: /ipns/gera2ld.space.

It's easy since we are deploying on Vercel, just create a vercel.json file in the project root:

{
  "headers": [
    {
      "source": "/:path(.*)",
      "headers" : [
        {
          "key" : "x-ipfs-path",
          "value" : "/ipns/gera2ld.space/:path"
        }
      ]
    }
  ]
}

Recap

Here is what we have done:

  • pack our files to a CAR
  • update DNS record for IPNS to work
  • upload the CAR to Web3.Storage to pin it on IPFS
  • tell Brave to suggest the IPFS entrance

By uploading the whole website as a CAR to the IPFS, we joined the world of distribution. It can significantly reduce server load and save bandwidth if you have a large audience and host media files.

© 2023