An Overview of the Solana SPL Token 2022 program (part 2) - The Transfer Hook

October 16 2023

In our previous post on Token2022 (find it here) we went over some of the simplest new features of the spl token 2022 program. This included extensions like transfer fees and interest rates, and how to implement those extensions on chain. In this post we will look at one of the most interesting, but more complex, new extensions, the transfer hook. This allows the developer to have a program on chain that is called into every time someone transfers a token from a particular mint. This could be used to manage fees for transferring NFTs, or could be used to instigate "events" in a game associated with transferring items where bandits might rob you and steal some of your items, or for updating metadata associated with a game item, such as evolving a pokemon like character on trade (We hope to have a final post in this series where we implement some of these examples).

There are three things that need to be considered when implementing the transfer hook:

• Setting up the extension in the mint

• Transferring tokens that make use of the extension

• Writing the actual transfer hook program following the required interface

All of these make use of some recently added functionality within the Solana ecosystem:

• Discriminators - A set of bytes that uniquely identify an instruction or struct
• POD (Plain Old Data) - Types which allow easy conversion from bytes to supported types
• Type-Length-Value (TLV) data structures - Used to store the set of additional accounts the transfer hook needs.
• ExtraAccountMeta - The structure that actually holds the extra accounts as a TLV type using Discriminators

Therefore, before getting into the implementation of the transfer hook itself, we will start by briefly describing these four features, and provide some context for how they are going to be used when setting up the transfer hook.

Discriminators

The discriminator type is implemented in the spl_discriminator crate which you can find here.The crate provides macros that allow the users to define a set of bytes to uniquely identify things like program instructions that will be called externally, or structs that can be deserialised using TLV data types. We will be using them in both these use cases in order to set up the transfer hook program.

A discriminator is a set of eight bytes that can be defined by the user in any way they see fit. The crate provides some useful macros that allow us to construct them directly from strings, so for example when defining the instructions that the transfer hook program requires, we can use the discriminator_hash_input macro to convert the provided strings into the discriminator for the trivial structs that will be used to denote the instructions:

    #[derive(SplDiscriminate)]
    #[discriminator_hash_input("spl-transfer-hook-interface:execute")]
    pub struct ExecuteInstruction;

    #[derive(SplDiscriminate)]
    #[discriminator_hash_input("spl-transfer-hook-interface:initialize-extra-account-metas")]
    pub struct InitializeExtraAccountMetaListInstruction;

Using this macro provides the struct with the SPL_DISCRIMINATOR_SLICE property, which we can use to distinguish between instructions. For example, when our transfer hook program is used, we must unpack the input that has been sent to the program to check what instruction has been called, and can use this property to do this:

    pub fn unpack(input: &[u8]) -> Result<Self, ProgramError> {
        // check first that the input is at least as long as the discriminator (8 bytes)
        if input.len() < ArrayDiscriminator::LENGTH {
            return Err(ProgramError::InvalidInstructionData);
        }
        let (discriminator, rest) = input.split_at(ArrayDiscriminator::LENGTH);
        Ok(match discriminator {
            ExecuteInstruction::SPL_DISCRIMINATOR_SLICE => {
                // unpack the amount that is being transferred
                let amount = rest
                    .get(..8)
                    .and_then(|slice| slice.try_into().ok())
                    .map(u64::from_le_bytes)
                    .ok_or(ProgramError::InvalidInstructionData)?;
                // return an execute instruction for 'amount'
                Self::Execute { amount }
            },
            // initialize extra account metas has no parameters, so just check the discriminator
            InitializeExtraAccountMetaListInstruction::SPL_DISCRIMINATOR_SLICE => {
                // if we have a match then return this instruction
                Self::InitializeExtraAccountMetaList
            },
            // otherwise return an error
            _ => return Err(ProgramError::InvalidInstructionData),
        })
    }

Likewise by using these discriminators, other programs can call our program and know how to specify which instruction they want to call.
Similarly when defining a TLV struct, one can add a discriminator to identify its type as in the example here, where the discriminator is just set to the integer value 1.

    #[repr(C)]
    #[derive(Clone, Copy, Debug, Default, PartialEq, Pod, Zeroable)]
    struct MyPodValue {
        data: [u8; 8],
    }
    impl SplDiscriminate for MyPodValue {
        const SPL_DISCRIMINATOR: ArrayDiscriminator = ArrayDiscriminator::new([1; ArrayDiscriminator::LENGTH]);
    }

Plain Old Data (POD)

In the above example you can see that we are making use of the 'Pod' attribute for struct MyPodValue. The POD data type is described in detail in the bytemuck crate here. Going into the weeds too much on this point is a bit out of scope for this post, but it is usefull to at least mention it's existence as you will see the code for the transfer hook makes use of things like PodBool, or PodAccountMeta types (i.e. take a type that you are familiar with and add Pod in front to make it new and mysterious). Essentially, by stating that a type has the Pod trait, you are stating that it satisfies a series of requirements, for example that the structure has no padding bytes, or that you can make a copy simply by copying the bytes. The default Bool type in Rust, and the AccountMeta type defined in the Solana crate do not satisfy these requirements, and so special versions have been implemented that do.

Type-Length-Value (TLV)

TLV structures are defined in the this crate, and are used in a wide range of applications. They provide a way of specifying the makeup of a slab of bytes, by specifying the type (8 bytes, using a discriminator), the length of the data (4 bytes), and then following that with the data itself as a slab of bytes of the given length. This way you can easily fill up an array of different data structures, each in the TLV format, and iterate through the structures to deserialise the data. The type then provides functionality to get the value (i.e to automatically deserialise the bytes to the specified format), or to get the discriminators to check the type. We will give an example of this below for the ExtraAccountMeta structure.

The ExtraAccountMeta struct

The ExtraAccountMeta struct is defined in the spl_tlv_account_resolution crate, and is used to hold all the extra account meta data for accounts involved in the transfer hook execution. In the simplest case this may not contain any accounts, or it might include accounts that contain meta data that should be updated as a result of the transfer. In our example below we will have a single account to decode which will contain the number of transfers that have occured for that mint. Each time a transfer occurs, our transfer hook program will increment that counter, and so everyone who tries to transfer one of the tokens for that mint will need to know what that account is, so that it can be included in the transfer instruction.

The combination of all the features discussed so far are used when unpacking this structure in the execute function of the transfer hook program, which we will discuss in more detail in the final part of this post:

    // get the data from the account that contains the extra account metas
    let data = extra_account_metas_info.data.borrow();
    // unpack this as a TLV state object
    let state = TlvStateBorrowed::unpack(&data[..]).unwrap();
    // get the ExtraAccountMetas object from the TLV state
    let extra_meta_list = ExtraAccountMetaList::unpack_with_tlv_state::<ExecuteInstruction>(&state)?;
    let extra_account_metas = extra_meta_list.data();

Setting Up The Mint

Setting up the transfer hook extension in the token mint starts in the same fashion as setting up any of the other extensions. You can find the full code for this section in the git repo for the previous blog post here here. We first simply have to invoke the initialize method for the transfer hook extension:

    // in processor.rs
    ...

    let config_init_idx =
    spl_token_2022::extension::transfer_hook::instruction::initialize(
        &spl_token_2022::ID,
        &token_mint_account_info.key,
        Some(*funding_account_info.key),
        Some(*transfer_hook_program_account.key),
    )
    .unwrap();

    invoke(
    &config_init_idx,
    &[
        token_program_account_info.clone(),
        token_mint_account_info.clone(),
        funding_account_info.clone(),
        transfer_hook_program_account.clone(),
    ],
    )?;

Here transfer_hook_program_account is the address of the deployed transfer hook program that will be called whenever a transfer occurs, which we will describe in the final section below. The extra step that is required to set up the transfer hook extension is to then call the function that will set up the meta data that describes the additional accounts that should be passed in order to use the transfer hook. We follow the example interface and call this InitializeExtraAccountMetas.

    // in processor.rs
    ...

    let mut account_metas = vec![
        // the account that will hold the extra meta data
        solana_program::instruction::AccountMeta::new(*transfer_hook_validation_account.key, false),
        // this mint account
        solana_program::instruction::AccountMeta::new(*token_mint_account_info.key, false),
        solana_program::instruction::AccountMeta::new(*funding_account_info.key, true),
        solana_program::instruction::AccountMeta::new_readonly(*system_program_account_info.key, false),
    ];

    let mut account_infos = vec![ 
        transfer_hook_validation_account.clone(),
        token_mint_account_info.clone(),
        funding_account_info.clone(),
        system_program_account_info.clone()
    ];

    // check if we added a mint data account
    // in our test this will hold the counter for the number of transfers for this mint
    if mint_data_option.is_some() {
        let mint_data_account_info = mint_data_option.unwrap();
        let mint_data_meta = solana_program::instruction::AccountMeta::new(*mint_data_account_info.key, false);
        account_metas.push(mint_data_meta);

        account_infos.push(mint_data_account_info.clone());
    }

    // pack is defined in instruction.rs and just sets the discriminator for this function
    let instruction_data = TransferHookInstruction::InitializeExtraAccountMetas.pack();

    let init_accounts_idx = solana_program::instruction::Instruction {
        program_id: *transfer_hook_program_account.key,
        accounts: account_metas,
        data: instruction_data,
    };

    invoke(
        &init_accounts_idx,
        &account_infos,
    )?;

This follows a pretty standard setup for a CPI call. We have an extra optional account, mint_data_account_info, which we will use in this post to count the number of transfers for each mint created by the program. This is a unique account per mint, and uses the mint address and the string "mint_data" as the seeds (this will be shown explicitly in the transfer hook program section).

Transferring Tokens

Transferring a token that has the transfer hook extension isn't quite as trivial as for the other extensions. The Solana blockchain requires that all accounts involved in a transaction are included as part of that transaction, but the default transfer_checked function doesn't know anything about our transfer hook program, or the data account that we will be modifying when that program is called. We will therefore to modify the instruction before we can invoke it.

We start by creating an instance of the transfer_checked instruction as normal, except that we have made it mutable. This means we can modify the set of accounts the instruction knows about afterwards:

    // in processor.rs
    ...

    // create a transfer_checked instruction
    // this needs to be mutable because we will manually add extra accounts
    let mut transfer_idx = spl_token_2022::instruction::transfer_checked(
        &spl_token_2022::id(),
        &source_token_account_info.key,
        &mint_account_info.key,
        &dest_token_account_info.key,
        &funding_account_info.key,
        &[&funding_account_info.key],
        metadata.amount,
        3,
    ).unwrap();

We can then add in the other accounts that are needed, these are:

• Out transfer hook program address
• The address of our ExtraAccountMeta
• The address of the account that will hold the data we update whenever a transfer occurs

    // add the three transfer hook accounts
    transfer_idx.accounts.push(AccountMeta::new_readonly(
        *hook_program_account_info.key,
        false,
    ));

    transfer_idx.accounts.push(AccountMeta::new_readonly(
        *validation_account_info.key,
        false,
    ));

    transfer_idx.accounts.push(AccountMeta::new(
        *mint_data_account_info.key,
        false,
    ));

    invoke(
        &transfer_idx,
        &[
            token_program_account_info.clone(),
            source_token_account_info.clone(),
            mint_account_info.clone(),
            dest_token_account_info.clone(),
            funding_account_info.clone(),
            hook_program_account_info.clone(),
            validation_account_info.clone(),
            mint_data_account_info.clone(),
        ],
    )?;

Once these have been pushed onto the accounts vector for the instruction we can just call invoke and the transfer will happen. Note that the spl_tlv_account_resolution crate does include some helper functions to do this more automatically (e.g. add_to_cpi_instruction) however in this case we found it simpler, and more instructive to just do it ourselves.

The Transfer Hook Program

In this final section we will implement the transfer hook program itself, the code for this section can be found in our git repo here. In our case we will be building two functions, both of which we have mentioned previously:

• Execute - the function called whenever a transfer occurs
• InitializeExtraAccountMetas - the function that will set up our ExtraAccountMeta data structure on chain

These two instructions are defined in instuctions.rs:

    pub enum TransferHookInstruction {
        Execute {
            amount: u64,
        },
        InitializeExtraAccountMetaList,
    }

InitializeExtraAccountMetas

We will start by going through InitializeExtraAccountMetas, the body of which is defined in process_initialize_extra_account_metas:

    // in processor.rs
    pub fn process_initialize_extra_account_metas<'a>(
        program_id: &Pubkey,
        accounts: &'a [AccountInfo<'a>],
    ) -> ProgramResult {
        
        // get and check the accounts
        ...

In our example the data account that will store the counter is optional, if we don't include it then the transfer hook program will still work but won't do anything. We therefore need to check if the account as been passed as an option. When creating the account that will hold the extra account meta data we will also need to know the total number of extra accounts to get the correct size, so we instantiate the mutable variable n_extra_accounts which we will increment if we have included the mint data account. If we have included the mint data account then we just get its size from a helper function defined in state.rs and then create the account as a PDA owned by the transfer hook program. Note that we use the mint address as part of the seed so that every mint that uses this program can have its own counter.

        let mut n_extra_accounts = 0;

        let mut extra_account_infos : Vec<ExtraAccountMeta> = vec![];
        // if we did pass a mint_data account then create that now
        if mint_data_option.is_some() {

            let mint_data_account_info = mint_data_option.unwrap();

            // check the account is what we expect
            let bump_seed = utils::check_program_data_account(
                mint_data_account_info, 
                program_id,
                vec![b"mint_data", &mint_info.key.to_bytes()], "mint_data".to_string()
            ).unwrap();
            
            // we need to create the mint_data account so that it can hold a u64 to count the number
            // of transfers
            // get_mint_data_size is defined in state.rs
            let data_size = state::get_mint_data_size();

            // create the account, this is defined in utils.rs and just makes the PDA
            Self::create_program_account(
                authority_info, 
                mint_data_account_info, 
                program_id, 
                bump_seed, 
                data_size, 
                vec![b"mint_data", &mint_info.key.to_bytes()]
            ).unwrap();

Once the account has been created we then define the ExtraAccountMeta structure that will store information about that account on chain. The ExtraAccountMeta structure has several different methods for instantiation depending on the type of account that is being saved. In this case we have made a PDA that is owned by our transfer hook program, and so we use the new_with_seeds function, which makes use of the Seed enum to save how to construct the address of the account. In our case we have aLiteral type seed that comes from the text "mint_data", and a seed that comes from an AccountKey (the mint account) which is defined using the index of that account as it was passed to this function (in our case that was account 1). Using those seeds we can instantiate the ExtraAccountMeta data and increment n_extra_accounts:

            ...

            // the ExtraAccountMeta needs to know this is a PDA and saves the seeds so that it can
            // check the account.  This is done using the Seed structure.
            let seed1 = Seed::Literal { bytes: b"mint_data".to_vec()};
            let seed2 = Seed::AccountKey { index: 1 };

            // create the ExtraAccountMeta for this account from the seeds
            let mint_account_meta = ExtraAccountMeta::new_with_seeds(&[seed1, seed2], false, true).unwrap();
            extra_account_infos.push(mint_account_meta);

            // increment the account counter
            n_extra_accounts = 1;
        }

That is most of the hard work done. Now we know how many extra accounts we have, we can determine the size of the account that will hold that data using the size_of function provided by ExtraAccountMetaList structure, and then create the account as normal:

        // given the number of extra accounts, get the size of the ExtraAccountMetaList
        let account_size = ExtraAccountMetaList::size_of(n_extra_accounts)?;

        let lamports = rent::Rent::default().minimum_balance(account_size);

        // create the account
        let ix = solana_program::system_instruction::create_account(
            authority_info.key,
            extra_account_metas_info.key,
            lamports,
            account_size as u64,
            program_id,
        );

        // Sign and submit transaction
        invoke_signed(
            &ix,
            &[authority_info.clone(), extra_account_metas_info.clone()],
            &[&[utils::EXTRA_ACCOUNT_METAS_SEED, mint_info.key.as_ref(), &[bump_seed]]],
        )?;

The last step of this function is then to actually initialize the data in this account using the ExtraAccountMeta data calculated previously (or using an empty array if we never passed the mint data account):

        // finally instantiate the data in the account from our extra_account_infos vec
        let mut data = extra_account_metas_info.try_borrow_mut_data()?;
        if  n_extra_accounts == 0 {
            ExtraAccountMetaList::init::<ExecuteInstruction>(&mut data, &[])?;
        }
        else {
            ExtraAccountMetaList::init::<ExecuteInstruction>(&mut data, &[extra_account_infos[0]])?;
        }

        Ok(())
    }

Execute

The final function to implement then is the Execute function, which is the thing that will actually be called every time a transfer occurs. In our case because this either does nothing (if no mint_data was passed when the mint was being created), or simply incrments a counter in a data account. As such it is really quite simple. The body of the function is defined in the process_execute function. Note that this gets passed the amount that has been transferred, however we don't use it at all in this example.

    // in processor.rs
    pub fn process_execute<'a>(
        program_id: &Pubkey,
        accounts: &'a [AccountInfo<'a>],
        _amount: u64,
    ) -> ProgramResult {

        let account_info_iter = &mut accounts.iter();

        // get and check accounts

To determine whether we have a mint_data account we check the account that we created in the previous step that contains the ExtraAccountMetaList. This involves creating a type-length-value object from the data, and then unpacking the bytes as the ExtraAccountMetaList object. In our case we only care if there are any accounts so we just check the length. If it is greater than zero we know that there must have been a mint data account passed, and so we enter that code block.

All we then have to do is deserialise the data in that account, increment the value by one, and then serialize it again and we are done!

        ...

        let data = extra_account_metas_info.data.borrow();

        let state = TlvStateBorrowed::unpack(&data[..]).unwrap();
        let extra_meta_list = ExtraAccountMetaList::unpack_with_tlv_state::<ExecuteInstruction>(&state)?;
        let extra_account_metas = extra_meta_list.data();

        if extra_account_metas.len() > 0 {
            let meta = extra_account_metas[0];

            let mint_data_account_info = next_account_info(account_info_iter)?;

            let mut player_state =
            state::MintData::try_from_slice(&mint_data_account_info.data.borrow()[..])?;

            player_state.count += 1;

            player_state.serialize(&mut &mut mint_data_account_info.data.borrow_mut()[..])?;
        }
        
        Ok(())
    }

Test Application

The application below lets you select any combination of the extensions we have discussed in this and the previous post and to mint some tokens using the Token-2022 program. In particular the transfer hook extension will use our on chain program to count the number of transfers that have been made using that mint and display that to the user.

CONNECT WALLET

Hopefully you have learnt something new about using the transfer hook with the Token-2022 program in this post. We will aim to follow this up at some point with some more interesting examples, such as evolving pokemon type characters in a game. If you don't to miss that then go ahead and follow us on Twitter to keep up to date with future posts!