Running and using Go-Ethereum

Most of the time, we have been using the Brownie development environment for our tests so far, and with it the Ganache Ethereum client that Brownie runs behind the scenes. For some applications, it is useful to have other clients at our disposal. The Go-Ethereum client (geth) is the most commonly used client at the time of writing, and today we take a slightly more detailed look at how to run and configure it.

Getting started

We have already installed the geth binary in a previous post, so I assume that the geth binary is still on your path. However, geth is evolving quickly – when I started this series, I used version 1.10.6, but in the meantime, 1.10.8 has been released which contains an important bugfix (for this vulnerability which we have analyzed in depth in a previous post ), so let us use this going forward. So please head over to the download page, get the archive for version 1.10.8 for your platform (here is the link for Linux AMD64), extract the archive, copy the geth binary to a location on your path and make it executable.

Geth stores blockchain data and a keystore in a data directory which can be specified via a command line flag. We will use ~/.ethereum, and it is easier to follow along if you start over with a clean setup, so if you have used geth before, you might want to delete the contents of the data directory before proceeding. To actually start the client, we can then use the command

geth --dev \
     --datadir=$HOME/.ethereum \
     --http \
     --http.corsdomain="*" \
     --http.vhosts="*" \
     --http.addr="0.0.0.0" \
     --http.api="eth,web3"

For the sake of convenience, I have put that command into a script in the tools directory of my repository, so if you have a clone of this directory, you could as well run ./tools/run_geth. Let us go through the individual flags that we have used and try to understand what they imply.

First, there is the dev flag. If you start geth with this flag, the client will actually not try to connect to any peers. Instead, it will create a local blockchain, with a genesis block which is created on the fly (here). In addition, geth will create a so-called developer account (or re-use an existing one). This account shows up at several places. It will be the first account in the keystore managed by geth and therefore the first account that the API method eth_accounts will return. This account (or rather the associated address) will also be used as the etherbase, i.e. as the address to which mined Ether will be credited. Finally, the genesis contains an allocation of 2256 – 9 Wei (the genesis block will also contain allocations for the nine pre-compiled contracts).

The next flag is the data directory, which we have already discussed. The following couple of flags are more interesting and configure the HTTP endpoint. Geth offers APIs over three different channels – HTTP, WebSockets (WS) and a local Unix domain socket (IPC). Whereas the IPC endpoint is enabled by default, the other two are disabled by default, and the http flag enables the HTTP endpoint.

The next three flags are important, as they determine who is allowed to access this API. First, http.address is the address on which the client will be listening. By default, this is the local host (i.e. 127.0.0.1), which implies that the client cannot be reached from the outside world. Especially, this will not work if you run geth inside a docker container or virtual machine. Specifying 0.0.0.0 as in the example above allows everybody on the local network to connect to your client – this is of course not a particularly secure setup, so modify this if you are not located on a secure and private network.

In addition, geth also uses the concept of a virtual host to validate requests. Recall that RFC7320 defines the HTTP header field Host which is typically used to allow different domains to be served by one web server running on one IP address only. This field is added by web browsers to requests that are the result of a navigation, but also the requests generated by JavaScript code running in the browser. When serving an incoming request, geth will validate the content of this header field against a list configured via the http.vhosts flag. This flag defaults to “localhost”. Thus, if you want to serve requests from outside, maybe from a web browser running on a different machine in your local network, you have to set this flag, either to the domain name of your server or using the wildcard “*” to accept incoming requests regardless of the value of the host header.

Finally, there is the CORS domain flag http.corsdomain. CORS is the abbreviation for cross-origin request surgery and refers to an attack which tries to access a server from JavaScript code loaded from a different domain. To prevent this sort of attack, browsers ask a server upfront before sending such a request whether the server will accept the request by submitting a so-called pre-flight request. When we develop our frontend later on, we will need to make sure that this pre-flight request is successful, so we need to include the domain from which we will load our JavaScript code to the list that we configure here, or, alternatively, also use a wildcard here. If you want to learn more about CORS, you might want to read this documentation on the Mozilla developer network.

The last flag that we use determines which of the APIs that geth offers will be made available via the HTTP endpoint. The most important API is the eth API, which contains all the methods that we need to submit and read transactions, get blocks or interact with smart contracts. In addition, it might be helpful to enable the debug API, which we will use a bit later when tracing transactions. There are also a few APIs which you almost never want to make available over the network like the personal API which allows you to access the accounts maintained by the client, or the admin API.

Using the geth console

We have just mentioned that there are some APIs which you would typically not want to make accessible via the API. Instead, you usually access these APIs via the IPC endpoint and the geth console. This is a JavaScript-based interactive console that allows you to invoke API methods and thus interact with a running geth client. To start the console, make sure that the geth client is running, open a separate terminal window and enter

geth attach ~/.ethereum/geth.ipc

Note that the second argument is the Unix domain socket that geth will create in its data directory. To see how the console works, let us play a bit with accounts. At the prompt, enter the following commands.

eth.blockNumber
eth.accounts
eth.getBlockByNumber(0)

The first command will return the block number for the block at the head of the chain. Currently, this is zero – we only have the genesis block, no other blocks. The second command displays the list of accounts managed by the node. You should see one account, which is the developer accounts mentioned earlier. The third command displays the genesis block, and you will see that the extra data also contains the developer account.

The accounts managed by the node can be controlled using the personal API. An important functionality of this API is that accounts can be locked, so that they can no longer be used. As an example, let us try to lock the developer account.

dev = eth.accounts[0]
personal.lockAccount(dev)

Unlocking the account again is a bit tricky, as this is not allowed while the HTTP endpoint is being served. So to unlock again, you will have to shutdown geth, start it again without the HTTP flags, attach again and execute the command

personal.unlockAccount(eth.accounts[0], "")

Note the second argument – this is the password that has been used to lock the account (at startup, geth creates the development account with an empty passphrase, alternatively a passphrase can be supplied using the —password command line flag).

Finally, let us see how to use the console to create additional accounts and transfer Ether to them.

dev = eth.accounts[0]
alice = personal.newAccount("secret")
value = web3.toWei(1, "ether")
gas = 21000
gasPrice = web3.toWei(1, "gwei")
txn = {
        from: dev, 
        to: alice, 
        gas: gas, 
        gasPrice: gasPrice, 
        value: value
}
hash = eth.sendTransaction(txn)
eth.getTransactionReceipt(hash)
eth.getBalance(alice)

You could now proceed like this and set up a couple of accounts, equipped with Ether, for testing purposes. To simplify this procedure, I have provided a script that sets up several test accounts – if you have cloned the repository, simply run it by typing

python3 tools/GethSetup.py

Geth and Brownie

What about Brownie? Do we have to say good-bye to our good old friend Brownie if we choose to work with geth? Fortunately the answer is no – Brownie is in fact smart enough and will automatically detect a running geth (in fact, a running Ethereum client) when it is started and use it instead of launching Ganache. Let us try this. Make sure that geth is running and start Brownie as usual.

brownie console

At this point, it is important that we have enabled the web3 API when starting geth, as Brownie uses the method web3_clientVersion to verify connectivity at startup. If everything works, Brownie will spit out a warning that the blockchain that it has detected has a non-zero length and greet you with the usual prompt.

Great, so let us transfer some Ether to a new account as we have done it before from the console to see that everything works.

dev = accounts[0]
bob = accounts.add()
dev.transfer(to=bob, amount=web3.toWei(1, "ether"))

Hmm…this does not look good. It appears that Brownie has created a transaction and sent it, but is now waiting for the receipt and does not receive it. To understand the problem, let us switch again to a different terminal and start the geth console again. At the console prompt, inspect the pending transactions by running

txpool

The output should show you that there is one pending transaction (which you can also inspect by using eth.getTransaction) which is not included in a block yet. If you look at this transaction for a second, you will find that there are two things that look suspicious. First, the gas price for the transaction is zero. Second, the gas limit is incredibly high. If you inspect the last block that has been mined, you will find that the gas limit is exactly the gas limit of the last block that has been mined successfully.

Why is this a problem? The gas limit for a new block is determined by geth aiming at a certain target value. At the moment, this target value is lower than the gas limit of the genesis block, meaning that geth will try to decrease the gas limit with each new block (the exact algorithm is here). Thus the gas limit for the new block that the miner tries to assemble is lower than that for the previous one and therefore lower than the gas limit of our transaction, so that the transaction will not fit into the block and the miner will ignore it.

Let us try to fix this. First, we need to replace our pending transaction. The easiest way to do this is to use the geth console. What we need to do is to get the transaction from the pool of pending transactions, correct the gas limit and increase the gas price, so that the miner will pick up this transaction instead of the previous one. We also set the value to zero, so that the transaction will effectively be cancelled.

txn = eth.pendingTransactions[0]
txn.gas = eth.estimateGas(txn)
txn.gasPrice = web3.toWei(1, "gwei")
txn.value = 0
eth.sendTransaction(txn)

Note that we did not change the nonce, so our transaction replaces the pending one. After a few seconds, Brownie should note that the transaction has been dropped and stop waiting for a receipt.

The reason for our problem is related to the way how Brownie determines the gas limit and gas price to be used for a transaction. When a transaction is created, Brownie tries to figure out a gas limit and gas price from the network configuration. For the gas limit, the default setting is “max”, which instructs Brownie to use the block gas limit of the latest blocks (which will be cached for one hour). For the gas price, the default is zero. To make Brownie work with geth, we need to adjust both settings. In the console, enter

network.gas_limit("auto")
network.gas_price("auto")

When you now repeat the commands above to transfer Ether, the transaction should go through. For versions of Brownie newer than version 1.15.2, however, you will receive an error message saying that the sleep method is not implemented by geth. The transaction will still work (the error comes from this line of the code which is executed in a separate thread initiated here), so the error message is only annoying, however you might want to downgrade to version 1.15.2 if you plan to work with Brownie and geth more often (it appears that the problem was introduced with this commit).

Note that the settings for the gas price and the gas limit that we have made enough will be lost when we restart Brownie. In order to make these changes permanent, you can add them to the configuration file for Brownie. Specifically, Brownie will, upon startup, load configuration data from a file called brownie-config.yaml. To set the gas price and the gas limit, create such a file with the following content

networks:
    default: development
    development:
        gas_limit: auto
        gas_price: auto

Here we adjust the configuration for the network development which we also declare as the default network and set the gas limit and the gas price to “auto”, instructing Brownie to determine a good approximation at runtime.

This closes our post for today. We have learned how to run geth in a local development environment, discussed the most important configuration options and seen how we can still use Brownie to work with transactions and contracts. In the next post, we will start to design and build our NFT wallet application and first try to understand the overall architecture.

Implementing and testing an ERC721 contract

In the previous posts, we have discussed the ERC721 standard and how metadata and the actual asset behind a token are stored. With this, we have all the ingredients in place to tackle the actual implementation. Today, I will show you how an NFT contract can be implemented in Solidity and how to deploy and test a contract using Brownie. The code for this post can be found here.

Data structures

As for our sample implementation of an ERC20 contract, let us again start by discussing the data structures that we will need. First, we need a mapping from token ID to the current owner. In Solidity, this would look as follows.

mapping (uint256 => address) private _ownerOf;

Note that we declare this structure as private. This does not affect the functionality, but for a public data structure, Solidity would create a getter function which blows up the contract size and thus makes deployment more expensive. So it is a good practice to avoid public data structures unless you really need this.

Now mappings in Solidity have a few interesting properties. In contrast to programming languages like Java or Python, Solidity does not offer a way to enumerate all elements of a mapping – and even if it did, it would be dangerous to use this, as loops like this can increase the gas usage of your contract up to the point where the block limit is reached, rendering it unusable. Thus we cannot simply calculate the balance of an owner by going through all elements of the above mapping and filtering for a specific owner. Instead, we maintain a second data structure that only tracks balances.

mapping (address => uint256) private _balances;

Whenever we transfer a token, we also need to update this mapping to make sure that it is in sync with the first data structure.

We also need a few additional mappings to track approvals and operators. For approvals, we again need to know which address is an approved recipient for a specific token ID, thus we need a mapping from token ID to address. For operators, the situation is a bit more complicated. We set up an operator for a specific address (the address on behalf of which the operator can act), and there can be more than one operator for a given address. Thus, we need a mapping that assigns to each address another mapping which in turn maps addresses to boolean values, where True indicates that this address is an operator for the address in the first mapping.

/// Keep track of approvals per tokenID
mapping (uint256 => address) private _approvals; 

/// Keep track of operators
 mapping (address => mapping(address => bool)) private _isOperatorFor;

Thus the sender of a message is an operator for an address owner if and only if _isOperatorFor[owner][msg.sender] is true, and the sender of a message is authorized to withdraw a token if and only if _approvals[tokenID] === msg.sender.

Burning and minting a token is now straightforward. To mint, we first check that the token ID does not yet exist. We then increase the balance of the contract owner by one and set the owner of the newly minted token to the contract owner, before finally emitting an event. To burn, we reverse this process – we set the current owner to the zero address and decrease the balance of the current owner. We also reset all approvals for this token. Note that in our implementation, the contract owner can burn all token, regardless of the current owner. This is useful for testing, but of course you would not want to do this in production – as a token owner, you would probably not be very amused to see that the contract owner simply burns all your precious token. As an aside, if you really want to fire up your own token in production, you would probably want to take a look at one of the available audited and thoroughly tested sample implementations, for instance by the folks at OpenZeppelin.

Modifiers

The methods to approve and make transfers are rather straightforward (with the exception of a safe transfer that we will discuss separately in a second). If you look at the code, however, you will spot a Solidity feature that we have not used before – modifiers. Essentially, a modifier is what Java programmers might know as an aspect – a piece of code that wraps around a function and is invoked before and after a function in your contract. Specifically, if you define a modifier and add this modifier to your function, the execution of the function will start off by running the modifier until the compiler hits upon the special symbol _ in the modifier source code. At this point, the code of the actual function will be executed, and if the function completes, execution continues in the modifier again. Similar to aspects, modifiers are useful for validations that need to be done more than once. Here is an example.

/// Modifier to check that a token ID is valid
modifier isValidToken(uint256 _tokenID) {
    require(_ownerOf[_tokenID] != address(0), _invalidTokenID);
    _;
}

/// Actual function
function ownerOf(uint256 tokenID) external view isValidToken(tokenID) returns (address)  {
    return _ownerOf[tokenID];
}

Here, we declare a modifier isValidToken and add it to the function ownerOf. If now ownerOf is called, the code in isValidToken is run first and verifies the token ID. If the ID is valid, the actual function is executed, if not, we revert with an error.

Safe transfers and the code size

Another Solidity feature that we have not yet seen before is used in the function _isContract. This function is invoked when a safe transfer is requested. Recall from the standard that a safe transfer needs to check whether the recipient is a smart contract and if yes, tries to invoke its onERC721Received method. Unfortunately, Solidity does not offer an operation to figure out whether an address is the address of a smart contract. We therefore need to use inline assembly to be able to directly run the EXTCODESIZE opcode. This opcode returns the size of the code of a given address. If this is different from zero, we know that the recipient is a smart contract.

Note that if, however, the code size is zero, the recipient might in fact still be a contract. To see why, suppose that a contract calls our NFT contract within its constructor. As the code is copied to its final location after the constructor has executed, the code size is still zero at this point. In fact, there is no better and fully reliable way to figure out whether an address is that of a smart contract in all cases, and even the ERC-721 specification itself states that the check for the onERC721Received method should be done if the code size is different from zero, accepting this remaining uncertainty.

Inline assembly is fully documented here. The code inside the assembly block is actually what is known as Yul – an intermediate, low-level language used by Solidity. Within the assembly code, you can access local variables, and you can use most EVM opcodes directly. Yul also offers loops, switches and some other high-level constructs, but we do not need any of this in your simple example.

Once we have the code size and know that our recipient is a smart contract, we have to call its onERC721Received method. The easiest way to do this in Solidity is to use an interface. As in other programming languages, an interface simply declares the methods of a contract, without providing an implementation. Interfaces cannot be instantiated directly. Given an address, however, we can convert this address to an instance of an interface, as in our example.

interface ERC721TokenReceiver
{
  function onERC721Received(address, address, uint256, bytes calldata) external returns(bytes4);
}

/// Once we have this, we can access a contract with this interface at 
/// address to
ERC721TokenReceiver erc721Receiver = ERC721TokenReceiver(to);
bytes4 retval = erc721Receiver.onERC721Received(operator, from, tokenID, data);

Here, we have an address to and assume that at this address, a contract implementing our interface is residing. We then convert this address to an instance of a contract implementing this interface, and can then access its methods.

Note that this is a pure compile-time feature – this code will not actually create a contract at the address, but will simply assume that a contract with that interface is present at the target location. Of course, we can, at compile time, not know whether this is really the case. The compiler can, however, prepare a call with the correct function signature, and if this method is not implemented, we will most likely end up in the fallback function of the target contract. This is the reason why we also have to check the return value, as the fallback function might of course execute successfully even if the target contract does not implement onERC721Received.

Implementing the token URI method

The last part of the code which is not fully straightforward is the generation of the token URI. Recall that this is in fact the location of the token metadata for a given token ID. Most NFT contracts that I have seen build this URI from a base URI followed by the token ID, and I have adapted this approach as well. The base URI is specified when we deploy the contract, i.e. as a constructor argument. However, converting the token ID into a string is a bit tricky, because Solidity does again not offer a standard way to do this. So you either have to roll your own conversion or use one of the existing implementations. I have used the code from this OpenZeppelin library to do the conversion. The code is not difficult to read – we first determine the number of digits that our number has by dividing by ten until the result is less than one (and hence zero – recall that we are dealing with integers) and then go through the digits from the left to the right and convert them individually.

Interfaces and the ERC165 standard

Our smart contract implements a couple of different interfaces – ERC-721 itself and the metadata extension. As mentioned above, interfaces are a compile-time feature. To improve type-safety at runtime, it would be nice to have a feature that allows a contract to figure out whether another contract implements a given interface. To solve this, EIP-165 has been introduced. This standard does two things.

First, it defines how a hash value can be assigned to an interface. The hash value of an interface is obtained by taking the 4-byte function selectors of each method that the interface implements and then XOR’ing these bytes. The result is a sequence of four bytes.

Second, it defines a method that each contract should implement that can be used to inquire whether a contract implements an interface. This method, supportsInterface, accepts the four-byte hash value of the requested interface as an argument and is supposed to return true if the interface is supported.

This can be used by a contract to check whether another contract implements a given interface. The ERC-721 standard actually mandates that a contract that implements the specification should also implement EIP-165. Our contract does this as well, and its supportsInterface method returns true if the requested interface ID is

  • 0x01ffc9a7, which corresponds to ERC-165 itself
  • 0x80ac58cd which is the hash value corresponding to ERC-721
  • 0x5b5e139f which is the hash value corresponding to the metadata extension

Testing, deploying and running our contract

Let us now discuss how we can test, deploy and run our contract. First, there is of course unit testing. If you have read my post on Brownie, the unit tests will not be much of a surprise. There are only two remarks that might be in order.

First, when writing unit tests with Brownie and using fixtures to deploy the required smart contracts, we have a choice between two different approaches. One approach would be to declare the fixtures as function scoped, so that they are run over and over again for each test case. This has the advantage that we start with a fresh copy of the contract for each test case, but is of course slow – if you run 30 unit tests, you conduct 30 deployments. Alternatively, we can declare the fixture as sessions-scoped. They will then be only executed once per test session, so that every test case uses the same instance of the contract under test. If you do this, be careful to clean up after each test case. A disadvantage of this approach, though, remains – if the execution of one test case fails, all test cases run after the failing test case will most likely fail as well because the clean up is skipped for the failed test case. Be aware of this and do not panic if all of a sudden almost all of your test cases fail (the -x switch to Brownie could be your friend if this happens, so that Brownie exits if the first test case fails).

A second remark is concerning mocks. To test a safe transfer, we need a target contract with a predictable behavior. This contract should implement the onERC721Received method, be able to return a correct or an incorrect magic value and allow us to check whether it has been called. For that purpose, I have included a mock that can be used for that purpose and which is also deployed via a fixture.

To run the unit tests that I have provided, simply clone my repository, make sure you are located in the root of the repository and run the tests via Brownie.

git clone https://github.com/christianb93/nft-bootcamp.git
cd nft-bootcamp
brownie test tests/test_NFT.py

Do not forget to first active your Python virtual environment if you have installed Brownie or any of the libraries that it requires in a virtual environment.

Once the unit tests pass, we can start the Brownie console which will, as we know, automatically compile all contracts in the contract directory. To deploy the contract, run the following commands from the Brownie console.

owner = accounts[0]
// Deploy - the constructor argument is the base URI
nft = owner.deploy(NFT, "http://localhost:8080/")

Let us now run a few tests. We will mint a token with ID 1, pick a new account, transfer the token to this account, verify that the transfer works and finally get the token URI.

alice = accounts[1]
nft._mint(1)
assert(owner == nft.ownerOf(1))
nft.transferFrom(owner, alice, 1)
assert(alice == nft.ownerOf(1))
nft.tokenURI(1)

I invite you to play around a bit with the various functions that the NFT contract offers – declare an operator, approve a transfer, or maybe test some validations. In the next few posts, we will start to work towards a more convenient way to play with our NFT – a frontend written using React and web3.js. Before we are able to work on this, however, it is helpful to expand our development environment a bit by installing a copy of geth, and this is what the next post will be about. Hope to see you there.

Using NFT metadata to safely store digital assets

If you read about NFTs for the first time, you might be under the impression that an NFT somehow stores a digital asset in the blockchain. In most of the cases, however, this is not true. If you have read my previous post, you will remember that in its essence, an NFT is nothing but a registry that maps assets, identified by a token ID, to its owners. So an NFT documents ownership, but does not even reference the asset itself in its basic form. But where is the asset itself? Answering this question is the purpose of todays post.

Storage options

Before getting into how and where digital assets managed by NFTs are typically stored, let us briefly discuss the storage options that we have at our disposal.

First, there is the Ethereum blockchain itself. Using the Ethereum state as long-term storage has a number of remarkable advantages – the data, once stored, is essentially immutable, easy to access and almost impossible to delete. However, there is also a major disadvantage which, in most cases, is actually a show-stopper – it is incredibly expensive.

Let us see what incredible really means and do some math. Writing a 32 byte word to storage by running the SSTORE opcode costs 20.000 units of gas. Thus 1 KB (1024 bytes, i.e. 32 words) cost 640.000 units of gas. The current gas price is at around 75 gWei, meaning that storing 1 kB would cost around 48 Mio. gWei, i.e. 0.05 Ether. At a price of 3000 USD / ETH, storing 1 kB of data would therefore cost roughly 150 USD. Storing a full digital asset, maybe an image, on-chain is therefore not really an option in most cases.

Alternatively, we could of course use any available centralized storage solution. This could be a web server that you or someone else operates, or one of the many available cloud-based storage solutions like Amazons S3, GCP storage or even Dropbox. These solutions are typically very affordable and might even have a free tier, and the data is backed up regularly and thus well protected against accidental data loss. These storage solutions are, however, centrally managed and operated. If someone mints an NFT and stores the digital asset that a token represents on a web server run on some AWS EC2 machine, there is a risk that this person either loses interest (or goes out of business) and simply shuts down the machine or deletes the asset at some point, or even manipulates the asset after the token has been minted. So data stored in these centralized solutions is typically not well protected against being tampered with and not durable.

Therefore, a third option is very popular in the NFT universe – decentralized peer-to-peer storage. In a previous post, we have already touched on IPFS. In addition to being decentralized, IPFS has the additional advantage of using content-based addressing. This simply means that the address under which data is accessible in the IPFS network is a hash of the data itself. If the data changes, so does the address, which implies that data stored in IPFS is effectively immutable – as long as you use the same address to access it, you can be sufficiently confident that the data is the same every time you access it. As a downside, however, the network can forget your data – everything that you store in IPFS will be distributed over many nodes, and a node can decide to drop your data in favor of data that is more frequently requested. Thus using IPFS to store real digital assets is only a good idea if you also find a node that pins your data (which, of course, again introduces a certain centralization).

NFT metadata

In many cases, you will not simply want to store the asset itself, but maybe other data somehow associated with the image, like a human-readable description, the artist or other attributes. To support this, most NFTs use a two-step approach. The token itself contains a link to a file with metadata in JSON format. One of the fields stored in this file is then a link (or, more precisely, an URI) pointing to the actual image.

In this way, all we have to store on-chain is the URI of the metadata. We can then choose any of the storage options that we have discussed for the metadata and the actual image. Note, however, that these choices of course will have an impact on the integrity of the image and the metadata, we will discuss this in a bit more depth in the next section.

As this scheme is very common, it has actually been standardized as part of the ERC-721 standard. In fact, the standard defines an (optional) method tokenURI which receives the token ID and returns a URI pointing to the metadata. The standard also proposes a scheme for the metadata, however, when we look at a few real-word example further below, we will find that none of them actually fully adheres to this standard – some leave out fields the description and most of them add non-standard fields like attributes.

The standard does not say anything about how the token URI is built from the token ID. It does, however, specify that the URI be unique, i.e. no two different token IDs should result in the same URI. A natural choice that we see in reference implementations is to use a base URI that is a property of the contract in combination with the token ID. This has the advantage that the token URI is automatically unique, and, assuming that the base URI is fixed over the lifetime of the contract, also stable. If the implementation allows a re-use of token IDs, this is especially useful as it avoids an attack where a token is burnt and then re-minted with the same ID but a different token URI (of course, such an attempt could be detected by listening on the created events, but could still be confusing for everybody who has stored the token ID).

Ensuring image integrity

How secure are the various storage options that we can use for metadata and image? If you place the metadata on a traditional, centrally managed web server, i.e. if you use a token URI like “http://my.site.com/token/…”, then, of course, whoever controls the domain “site.com” is able to change the metadata or even redirect queries to a different server. One approach to protect against this is to somehow add a cryptographic hash of the metadata to the token as well, so that this hash is stored on-chain, and can be used to validate the metadata.

This approach has been standardized in EIP-2477. This standard (which, actually, is only a draft at the time of writing) adds the method tokenURIIntegrity which is supposed to return a hash value of the metadata and the algorithm used to create the hash. It appears, however, that this is not yet widely adopted, and in fact, none of the examples that we will analyze in the next section offers this method.

So if you really want to make sure that the metadata is not changed, you can either use this approach or a content-based medium like IPFS or a non-standard extension (the token URI could, for instance, contain a hash of the metadata even if it points to a traditional web server).

What about the image itself? The image URI is part of the metadata, but of course whoever controls the storage system could replace the image itself by some other content. Again, there are several options to mitigate this risk – you could use content-addressable storage for the image as well, add a cryptographic hash of the image to the metadata or add a cryptographic hash value of the image to the token data.

The choice is yours, and probably depends on the nature of the asset and the infrastructure which is at your disposal. As a guidance, however, it is of course helpful to see how existing contracts out there in the wild are approaching this, and this is what we will do now.

Real world use cases

To get an idea for some real-world use cases, let us head over to the OpenSea NFT market place and look at some NFTs. Our first example is the ArtBlocks collection. Technically, this is an ERC721 contract which implements a couple of additional methods, among them the methods of the metadata extensions. Let us take a look at one specific item in this collection, say #507 which has the token ID 21000507. We can use Etherscan to query the tokenURI method for this token ID.

As a result, you should get the following URI back.

https://api.artblocks.io/token/21000507

So the token URI is built from a base URI and the token ID (if you look at the source code of the contract, you will find that theoretically, the contract also allows to optionally use IPFS, but at least for this specific token, this option has apparently not been chosen). If you navigate to this URL, you will obtain a JSON-formatted file which contains a couple of attributes, like the artist, a description, a hash value and a link to the image.

Taking a look at the description on the project homepage tells us that the hash value is, in this case, not a hash of the image itself, but a seed from which the image can be recreated at any point in time. This seed is also stored on-chain, in a public mapping called tokenIdToHash. As Solidity automatically generates getter functions for public variables, you can query this mapping using Etherscan and confirm that the hash string in the metadata and the entry in the mapping match. The same is true for the script itself, thus even though the image is stored at a central location, the data needed to reconstruct the image and verify its integrity is completely stored on-chain.

As a second example, let us look at the 0N1 Force collection. This is a collection of 7.777 randomly-generated virtual characters which are traded as an NFT. Again, this is an NFT according to the ERC721 standard that also realizes the metadata extension. This time, however, both metadata and image are stored on IPFS.

To illustrate this, we will use character 493 out of this collection. In order to retrieve the token URI, again head over to the contract on Etherscan, select the tokenURI method, enter the token ID 493 and hit the “Query” button. As a result you should see

ipfs://QmXgSuLPGuxxRuAana7JdoWmaS25oAcXv3x2pYMN9kVfg3/493

This is a link into the IPFS file system. To get the metadata behind, you either need to install an IPFS node yourself or – much easier – use one of the available web-based IPFS gateways. Let us use ipfs.io itself, which makes the content of the metadata we are after available at this URL. The metadata file which we obtain this way again contains a few attributes and an image link, again pointing to an IPFS CID which contains the actual image (be careful, accessing this link is slow and might result in HTML error 429 a few times). Thus we are in the scheme presented above – the CID for the metadata protects the integrity of the metadata, the metadata contains the CID of the image and therefore the integrity of the image is protected as well.

Finally, let us look at a more classical example – the “Now you are free” painting from the first Damien Hirst collection “Currency”. From a technical point, this again follows the same pattern – the contract has a tokenURI method, querying this for the token ID 7813 returns a link to the IPFS metadata and this in turn contains a link to the actual image, both on IPFS. According to the official project homepage, however, each of the images initially exists twice – purely digital and as a physical painting. During a certain period of time after minting the NFT, the owner can then decide to either exchange the NFT for the physical painting – in which case the token will be burnt – or keep the token – in which case the physical painting will be destroyed.

This closes our post for today. In this post and the previous post, we have covered the ERC-721 token standard in sufficient detail to be ready to get our hands dirty and dive into the actual implementation. To implement an NFT, we will need some additional features of Solidity which we will discuss in the next post.