Base models

Base models describe how cryptoassets.core handles any cryptocurrency on the database level. SQLAlchemy library is used for modeling.

Models are abstract and when you instiate a new cryptocurrency, you inherit from the base classes and set the cryptocurrency specific properties.

Models also specify the core API how to interact with cryptoassets.core

See how to get started interacting with models.

For more information, see coin documentation and how to extend the framework with your own altcoins.

Cryptoasset registry

All running cryptoassets are maintained in a coin registry.

Each cryptoasset provides its own Wallet SQLAlchemy model and backend instance which is used to communicate with the network of the cryptoasset.

class cryptoassets.core.coin.registry.CoinModelDescription(coin_name, wallet_model_name, address_model_name, account_model_name, transaction_model_name, network_transaction_model_name, address_validator)[source]

Describe one cryptocurrency data structures: what SQLAlchemy models and database tables it uses.

The instance of this class is used by cryptoassets.core.models.CoinDescriptionModel to build the model relatinoships and foreign keys between the tables of one cryptoasset.

Create the description with fully dotted paths to Python classes.

Parameters:coin_name – Name of this coin, lowercase acronym
Wallet

Get wallet model class.

Address

Get address model class.

Account

Get account model class.

NetworkTransaction

Get network transaction model class.

Transaction

Get transaction model class.

class cryptoassets.core.coin.registry.Coin(coin_description, backend=None, max_confirmation_count=15, testnet=False)[source]

Describe one cryptocurrency setup.

Binds cryptocurrency to its backend and database models.

We also carry a flag if we are running in testnet or not. This affects address validation.

Create a binding between asset models and backend.

Parameters:
backend = None

Subclass of cryptoassets.core.backend.base.CoinBackend.

name = None

Lowercase acronym name of this asset

max_confirmation_count = None

This is how many confirmations tools.confirmationupdate tracks for each network transactions, both incoming and outgoing, until we consider it “closed” and stop polling backend for updates.

address_model

Property to get SQLAlchemy model for address of this cryptoasset.

Subclass of cryptoassets.core.models.GenericAddress.

transaction_model

Property to get SQLAlchemy model for transaction of this cryptoasset.

Subclass of cryptoassets.core.models.GenericTransaction.

account_model

Property to get SQLAlchemy model for account of this cryptoasset.

Subclass of cryptoassets.core.models.GenericAccount.

wallet_model

Property to get SQLAlchemy model for account of this cryptoasset.

Subclass of cryptoassets.core.models.GenericWallet.

network_transaction_model

Property to get SQLAlchemy model for account of this cryptoasset.

Subclass of cryptoassets.core.models.GenericWallet.

validate_address(address)[source]

Check the address validy against current network.

Returns:True if given address is valid.
class cryptoassets.core.coin.registry.CoinRegistry[source]

Holds data of set up cryptocurrencies.

Usually you access this through cryptoasssets.core.app.CryptoassetsApp.coins instance.

Example:

cryptoassets_app = CryptoassetsApp()
# ... setup ...

bitcoin = cryptoassets_app.coins.get("btc)

print("We are running bitcoin with backend {}".format(bitcoin.backend))
all()[source]

Get all registered coin models.

Returns:List of tuples(coin name, Coin)
get(name)[source]

Return coin setup data by its acronym name.

Parameters:name – All lowercase, e.g. btc.

Default models

Default cryptocurrency names and their models.

cryptoassets.core.coin.defaults.COIN_MODEL_DEFAULTS = {'btc': 'cryptoassets.core.coin.bitcoin.models', 'aby': 'cryptoassets.core.coin.applebyte.models', 'doge': 'cryptoassets.core.coin.dogecoin.models', 'ltc': 'cryptoassets.core.coin.litecoin.models'}

This is the default mapping between the three-letter coin acronyms and their SQLAlchemy model presentations. If you want to use your own database models you can override any of these in your configuration.

Model API conventions

The following conventions are followed in the model API

Model discovery

  • Abstract base classes are called GenericXxx like GenericWallet.
  • Actual class implementation is in coin module, e.g. cryptoassets.core.coin.bitcoin.models.BitcoinWallet.
  • You do not access the model classes directly, but through configured assets registry. E.g. to get a hold of BitcoinWallet class you do Wallet = cryptoassets_app.coins.get("btc").coin_model.
  • The usual starting point for the calls is to get or create cryptoassets.core.models.GenericWallet instance. Check out cryptoassets.core.models.GenericWallet.get_or_create_by_name().

Session lifecycle

  • API tries to use the SQLAlchemy database session of the object if possible: Session.object_session(self). If not, session must be explicitly given and you get your session inside a helper closure function decorated by cryptoassets.core.utils.conflictresolver.ConflictResolver.managed_transaction(). This way we guarantee graceful handling of transaction conflicts.
  • API never does session.flush() or session.commit()
  • API will do session.add() for newly created objects

Model classes

Below are the base classes for models. All cryptoassets have the same API as described these models.

Account

class cryptoassets.core.models.GenericAccount[source]

An account within the wallet.

We associate addresses and transactions to one account.

The accountn can be owned by some user (user’s wallet), or it can be escrow account or some other form of automatic transaction account.

The transaction between the accounts of the same wallet are internal and happen off-blockhain.

A special account is reserved for network fees caused by outgoing transactions.

NETWORK_FEE_ACCOUNT = 'Network fees'

Special label for an account where wallet will put all network fees charged by the backend

id = Column(None, Integer(), table=None, primary_key=True, nullable=False)

Running counter used in foreign key references

name = Column(None, String(length=255), table=None)

Human-readable name for this account

created_at = Column(None, DateTime(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f08b61e4950>))

When this account was created

updated_at = Column(None, DateTime(), table=None, onupdate=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f08b61e49d8>))

Then the balance was updated, or new address generated

balance = Column(None, Numeric(precision=21, scale=8), table=None, nullable=False, default=ColumnDefault(0))

Available internal balance on this account NOTE: Accuracy checked for bitcoin only

pick_next_receiving_address_label()[source]

Generates a new receiving address label which is not taken yet.

Some services, like block.io, requires all receiving addresses to have an unique label. We use this helper function in the situations where it is not meaningful to hand-generate labels every time.

Generated labels are not user-readable, they are only useful for admin and accounting purposes.

get_unconfirmed_balance()[source]

Get the balance of this incoming transactions balance.

TODO: Move to its own subclass

TODO: Denormalize unconfirmed balances for faster look up?

Returns:Decimal

Address

class cryptoassets.core.models.GenericAddress(**kwargs)[source]

The base class for cryptocurrency addresses.

The address can represent a

  • Receiving address in our system. In this case we have account set to non-NULL.
  • External address outside our system. In this account is set to NULL. This address has been referred in outgoing broadcast (XXX: subject to change)

We can know about receiving addresses which are addresses without our system where somebody can deposit cryptocurrency. We also know about outgoing addresses where somebody has sent cryptocurrency from our system. For outgoing addresses wallet reference is null.

Warning

Some backends (block.io) enforce that receiving address labels must be unique across the system. Other’s don’t. Just bear this in mind when creating address labels. E.g. suffix them with a timetamp to make them more unique.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

id = Column(None, Integer(), table=None, primary_key=True, nullable=False)

Running counter used in foreign key references

address = Column(None, String(length=127), table=None, nullable=False)

The string presenting the address label in the network

label = Column(None, String(length=255), table=None)

Human-readable label for this address. User for the transaction history listing of the user.

balance = Column(None, Numeric(precision=21, scale=8), table=None, nullable=False, default=ColumnDefault(0))

Received balance of this address. Only confirmed deposits count, filtered by GenericConfirmationTransaction.confirmations. For getting other balances, check get_balance_by_confirmations(). NOTE: Numeric Accuracy checked for Bitcoin only ATM

archived_at = Column(None, DateTime(), table=None)

Archived addresses are no longer in active incoming transaction polling and may not appear in the user wallet list

get_received_transactions(external=True, internal=True)[source]

Get all transactions this address have received, both internal and external deposits.

get_balance_by_confirmations(confirmations=0, include_internal=True)[source]

Calculates address’s received balance of all arrived incoming transactions where confirmation count threshold is met.

By default confirmations is zero, so we get unconfirmed balance.

Note

This is all time received balance, not balance left after spending.

TODO: Move to its own subclass

Parameters:confirmations – Confirmation count as threshold

Transaction

class cryptoassets.core.models.GenericTransaction(**kwargs)[source]

A transaction between accounts, incoming transaction or outgoing transaction.

Transactions can be classified as following:

  • Deposit: Incoming, external, transaction from cryptocurrency network.

    • Has network_transaction set.
    • Has receiving_account set.
    • No sending_account
  • Broadcast: Outgoign, external, transaction to cryptocurrency network.

    • Has network_transaction set.
    • Has receiving_account set.
    • No receiving_account
  • Internal transactions

    • Which are not visible outside our system.
    • have both sending_account and receiving_account set.
    • network_transaction is null
    • Internal transactions can be further classified as: ìnternal (normal between accounts), balance_import (initial wallet import to system) and network_fee (fees accounted to the network fee account when transaction was broadcasted)

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

id = Column(None, Integer(), table=None, primary_key=True, nullable=False)

Running counter used in foreign key references

created_at = Column(None, DateTime(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f08b61ea400>))

When this transaction become visible in our database

credited_at = Column(None, DateTime(), table=None)

When the incoming transaction was credited on the account. For internal transactions it is instantly. For external transactions this is when the confirmation threshold is exceeded.

processed_at = Column(None, DateTime(), table=None)

When this transaction was processed by the application. For outgoing transactions this is the broadcasting time. For incoming transactions, your application may call mark_as_processed to mark it has handled the transaction.

amount = Column(None, Numeric(precision=21, scale=8), table=None)

Amount in the cryptocurrency minimum unit Note: Accuracy checked for Bitcoin only

state = Column(None, Enum('pending', 'broadcasted', 'incoming', 'processed', 'internal', 'network_fee', 'balance_import', name='transaction_state'), table=None, nullable=False)

Different states this transaction can be

pending: outgoing transaction waiting for the broadcast

broadcasted: outgoing transaction has been sent to the network

incoming: we see the transaction incoming to our system, but the confirmation threshold is not exceeded yet processed: the application marked this transaction as handled and cryptoassets.core stops trying to notify your application about the transaction

internal: This transaction was between the accounts within one of our wallets

network_fee: When the transaction has been broadcasted, we create an internal transaction to account the occured network fees

label = Column(None, String(length=255), table=None)

Human readable label what this transaction is all about. Must be unique for each account

can_be_confirmed()[source]

Return if the transaction can be considered as final.

txid

Return txid of associated network transaction (if any).

Shortcut for self.network_transaction.txid.

NetworkTransaction

class cryptoassets.core.models.GenericNetworkTransaction(**kwargs)[source]

A transaction in cryptocurrencty networkwhich is concern of our system.

External transactions can be classified as

  • Deposits: incoming transactions to our receiving addresses
  • Broadcasts: we are sending out currency to the network

If our intenal transaction (cryptoassets.core.models.Transaction) has associated network transaction, it’s transaction.network_transaction reference is set. Otherwise transactions are internal transactions and not visible in blockchain.

Note

NetworkTransaction does not have reference to wallet. One network transaction may contain transfers to many wallets.

Handling incoming deposit transactions

For more information see cryptoassets.core.backend.transactionupdater and cryptoassets.core.tools.confirmationupdate.

Broadcasting outgoing transactions

Broadcast constructs an network transaction and bundles any number of outgoing pending transactions to it. During the broadcast, one can freely bundle transactions together to lower the network fees, or mix transactions for additional privacy.

Broadcasts are constructed by Cryptoassets helper service which will periodically scan for outgoing transactions and construct broadcasts of them. After constructing, broadcasting is attempted. If the backend, for a reason or another, fails to make a broadcast then this broadcast is marked as open and must be manually vetted to succeeded or failed.

For more information see cryptoassets.core.tools.broadcast.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

id = Column(None, Integer(), table=None, primary_key=True, nullable=False)

Running counter used in foreign key references

created_at = Column(None, DateTime(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f08b61efbf8>))

When this transaction become visible in our database

txid = Column(None, String(length=255), table=None)

Network transaction has associated with this transaction. E.g. Bitcoin transaction hash.

transaction_type = Column(None, Enum('deposit', 'broadcast', name='network_transaction_type'), table=None, nullable=False)

Is this transaction incoming or outgoing from our system

opened_at = Column(None, DateTime(), table=None)

When broadcast was marked as outgoing

closed_at = Column(None, DateTime(), table=None)

When broadcast was marked as sent

classmethod get_or_create_deposit(session, txid)[source]

Get a hold of incoming transaction.

Returns:tuple(Instance of cryptoassets.core.models.GenericNetworkTransaction., bool created)
class cryptoassets.core.models.GenericConfirmationNetworkTransaction(**kwargs)[source]

Mined transaction which receives “confirmations” from miners in blockchain.

This is a subtype of GenericNetworkTransaction with confirmation counting abilities.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

confirmations = Column(None, Integer(), table=None, nullable=False, default=ColumnDefault(-1))

How many miner confirmations this tx has received. The value is -1 until the transaction is succesfully broadcasted, after which is it 0

confirmation_count = 3

How many confirmations to wait until the transaction is set as confirmed. TODO: Make this configurable.

can_be_confirmed()[source]

Does this transaction have enough confirmations it could be confirmed by our standards.

Wallet

class cryptoassets.core.models.GenericWallet[source]

A generic wallet implemetation.

Inside the wallet there is a number of accounts.

We support internal transaction between the accounts of the same wallet as off-chain transactions. If you call send()``for the address which is managed by the same wallet, an internal transaction is created by ``send_internal().

id = Column(None, Integer(), table=None, primary_key=True, nullable=False)

Running counter used in foreign key references

name = Column(None, String(length=255), table=None)

The human-readable name for this wallet. Only used for debugging purposes.

created_at = Column(None, Date(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f08b61ead08>))

When this wallet was created

updated_at = Column(None, Date(), table=None, onupdate=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f08b61ead90>))

Last time when the balance was updated or new receiving address created.

balance = Column(None, Numeric(precision=21, scale=8), table=None)

The total balance of this wallet in the minimum unit of cryptocurrency NOTE: accuracy checked for Bitcoin only

classmethod get_by_id(session, wallet_id)[source]

Returns an existing wallet instance by its id.

Returns:Wallet instance
classmethod get_or_create_by_name(name, session)[source]

Returns a new or existing instance of a named wallet.

Returns:Wallet instance
create_account(name)[source]

Create a new account inside this wallet.

Returns:GenericAccout object
get_or_create_network_fee_account()[source]

Lazily create the special account where we account all network fees.

This is for internal bookkeeping only. These fees MAY be charged from the users doing the actual transaction, but it must be solved on the application level.

create_receiving_address(account, label=None, automatic_label=False)[source]

Creates a new receiving address.

All incoming transactions on this address are put on the given account.

The notifications for transctions to the address might not be immediately available after the address creation depending on the backend. For example, with block.io you need to wait some seconds before it is safe to send anything to the address if you wish to receive the wallet notification.

Parameters:
  • account – GenericAccount object
  • label – Label for this address - must be human-readable
Returns:

GenericAddress object

get_or_create_external_address(address)[source]

Create an accounting entry for an address which is outside our system.

When we send out external transactions, they go to these address entries. These addresses do not have wallet or account connected to our system.

Parameters:address – Address as a string
send(from_account, receiving_address, amount, label, force_external=False, testnet=False)[source]

Send the amount of cryptocurrency to the target address.

If the address is hosted in the same wallet do the internal send with cryptoassets.core.models.GenericWallet.send_internal(), otherwise go through the public blockchain with cryptoassets.core.models.GenericWallet.send_external().

Parameters:
  • from_account – The account owner from whose balance we
  • receiving_address – Receiving address as a string
  • amount – Instance of Decimal
  • label – Recorded text to the sending wallet
  • testnet – Assume the address is testnet address. Currently not used, but might affect address validation in the future.
  • force_external – Set to true to force the transaction go through the network even if the target address is in our system.
Returns:

Transaction object

add_address(account, label, address)[source]

Adds an external address under this wallet, under this account.

There shouldn’t be reason to call this directly, unless it is for testing purposes.

Parameters:
  • account – Account instance
  • address – Address instance
get_account_by_address(address)[source]

Check if a particular address belongs to receiving address of this wallet and return its account.

This does not consider bitcoin change addresses and such.

Returns:Account instance or None if the wallet doesn’t know about the address
get_pending_outgoing_transactions()[source]

Get the list of outgoing transactions which have not been associated with any broadcast yet.

get_receiving_addresses(archived=False)[source]

Get all receiving addresses for this wallet.

This is mostly used by the backend to get the list of receiving addresses to monitor for incoming transactions on the startup.

Parameters:expired – Include expired addresses
get_deposit_transactions()[source]

Get all deposit transactions to this wallet.

These are external incoming transactions, both unconfirmed and confirmed.

Returns:SQLAlchemy query of Transaction model
get_active_external_received_transcations()[source]

Return unconfirmed transactions which are still pending the network confirmations to be credited.

Returns:SQLAlchemy query of Transaction model
refresh_account_balance(account)[source]

Refresh the balance for one account.

If you have imported any addresses, this will recalculate balances from the backend.

TODO: This method will be replaced with wallet import.

TODO: This screws ups bookkeeping, so DON’T call this on production. It doesn’t write fixing entries yet.

Parameters:account – GenericAccount instance
send_internal(from_account, to_account, amount, label, allow_negative_balance=False)[source]

Tranfer currency internally between the accounts of this wallet.

Parameters:
  • from_account – GenericAccount
  • to_account – GenericAccount
  • amount – The amount to transfer in wallet book keeping unit
send_external(from_account, to_address, amount, label, testnet=False)[source]

Create a new external transaction and put it to the transaction queue.

When you send cryptocurrency out from the wallet, the transaction is put to the outgoing queue. Only after you broadcast has been performed (cryptoassets.core.tools.broadcast) the transaction is send out to the network. This is to guarantee the system responsiveness and fault-tolerance, so that outgoing transactions are created even if we have temporarily lost the connection with the cryptocurrency network. Broadcasting is usually handled by cryptoassets helper service.

Parameters:
  • from_account – Instance of cryptoassets.core.models.GenericAccount
  • to_address – Address as a string
  • amount – Instance of Decimal
  • label – Recorded to the sending wallet history
  • testnet – to_address is a testnet address
Returns:

Instance of cryptoassets.core.models.GenericTransaction

charge_network_fees(broadcast, fee)[source]

Account network fees due to transaction broadcast.

By default this creates a new accounting entry on a special account (GenericAccount.NETWORK_FEE_ACCOUNT) where the network fees are put.

Parameters:
  • txs – Internal transactions participating in send
  • txid – External transaction id
  • fee – Fee as the integer
refresh_total_balance()[source]

Make the balance to match with the actual backend.

This is only useful for send_external() balance checks. Actual address balances will be out of sync after calling this (if the balance is incorrect).

deposit(ntx, address, amount, extra=None)[source]

Informs the wallet updates regarding external incoming transction.

This method should be called by the coin backend only.

Write the transaction to the database. Notify the application of the new transaction status. Wait for the application to mark the transaction as processed.

Note that we may receive the transaction many times with different confirmation counts.

Parameters:
  • ntx – Associated cryptoassets.core.models.NetworkTransaction
  • address – Address as a string
  • amount – Int, as the basic currency unit
  • extra – Extra variables to set on the transaction object as a dictionary. (Currently not used)
Returns:

tuple (Account instance, new or existing Transaction object, credited boolean)

mark_transaction_processed(transaction_id)[source]

Mark that the transaction was processed by the client application.

This will stop retrying to post the transaction to the application.

Validation

Coin models support pluggable address validators.

We provide some validators just to make sure we don’t write bad outgoing transactions to our database.

class cryptoassets.core.coin.validate.AddressValidator[source]

Define address validation interface.

You should not call this directly, instead use cryptoassets.core.coin.registry.Coin.validate_address().

validate_address(address, testnet)[source]
Parameters:
  • address – Address as a string
  • testnet – We are in testnet
Returns:

True if the address is valid

class cryptoassets.core.coin.validate.HashAddresValidator[source]

Check that hash in the address is good.

Does not do extensive checks like address type, etc. one could do with pycoin.

http://rosettacode.org/wiki/Bitcoin/address_validation

class cryptoassets.core.coin.validate.NetworkCodeAddressValidator(netcode, testnetcode)[source]

Check if Bitcoin style address is valid using pycoin library.

XXX: Issues, could not get working.