Application and API service support

Introduction

cryptoassets.core can operate on raw cryptocurrency server daemon. Alternative you can choose one of the API services in the case you do not have the budget to run the full cryptocurrency node.

One instance of cryptoassets.core supports multiple backends. E.g. you can run application doing both Bitcoin and Dogecoin at the same time. However one backend can be enabled for one cryptoasset at a time.

After the backend has been set up you rarely interact it with directly. Instead, you use model APIs and cryptoassets helper service takes care of cryptoassets operations.

Running your cryptocurrency daemon (bitcoind)

Pros

  • You have 100% control of assets
  • You are not dependend of any vendor (downtime, asset seizure)

Cons

  • Running bitcoind requires root server access, 2 GB of RAM and 25 GB of disk space minimum and cannot be done on low budget web hosting accounts
  • You need to have sysadmin skills and be aware of server security

Using API service

Pros

  • Easy to set up
  • Works with even low budget hosting

Cons

  • Increases attack surface
  • If the service provider goes out of business you might lose your hot wallet assets

Configuration

The backend is configured separate for each cryptocurrency (BTC, Doge) and registered in cryptoassets.backend.registry. Each backend takes different initialization arguments like API keys and passwords. You usually set up these in cryptoassets.core config.

The active backend configuration can be read through cryptoassets.core.coin.registry.CoinRegistry. Bindings between the backends and the cryptocurrenct are described by cryptoassets.core.coin.registry.Coin class.

Backends

bitcoind

bitcoind and bitcoind-derivate backend. Interact directly with bitcoind service running on your own server.

Because most bitcoin forks have the same JSON-RPC API as the original bitcoind, you can use this backend for having service for most bitcoind-derived altcoins.

You must configure bitcoind on your server to work with cryptoassets.core. This happens by editing bitcoin.conf.

Example bitcoin.conf:

# We use bitcoin testnet, not real bitcoins
testnet=1

# Enable JSON-RPC
server=1

# Username and password
rpcuser=foo
rpcpassword=bar

rpctimeout=5
rpcport=8332

# This must be enabled for gettransaction() to work
txindex=1

# Send notifications to cryptoassetshelper service over HTTP
walletnotify=curl --data "txid=%s" http://localhost:28882

Note

You need to install curl on your server too (sudo apt install curl)

The backend configuration takes following parameters.

param class:Always cryptoassets.core.backend.bitcoind.Bitcoind
param url:Bitcoind connection URL with username and password (rpcuser and rpcassword in bitcoin config) for AuthServiceProxy. Usually something like http://foo:bar@127.0.0.1:8332/
param walletnotify:
 Dictionary of parameters to set up walletnotify handler.
param timeout:Timeout for JSON-RPC call. Default is 15 seconds. If the timeout occurs, the API operation can be considered as failed and the bitcoind as dead.

Wallet notifications

Wallet notifications (or, short, walletnotify) is the term used by cryptoasset.core to describe how backend communicates back to cryptoassets helper service. It’s named after bitcoind walletnotify option.

  • You can setup different wallet notify method depending if you run daemon application locally, on a remote server or you use some API service
  • In theory, you could mix and match backends and wallet notifications methods. But just stick to what is recommended for the backend recommends.
  • Each cryptoasset require its own notification channel (named pipe, HTTP server port)

HTTP webhook for bitcoind

Handle walletnofify notificatians from bitcoind through curl / wget HTTP request.

This is useful in cases where bitcoind or alike is running on a remote server and you wish to receive walletnotifications from there. In this case, you can set up SSH tunnel and forward the locally started HTTP wallet notify listener to the bitcoind server.

Creates a HTTP server running in port 28882 (default). To receive a new transaction notification do a HTTP POST to this server:

curl --data "txid=%s" http://localhost:28882

E.g. in bitcoind.conf:

walletnotify=curl --data "txid=%s" http://localhost:28882
Options
param class:Always cryptoassets.core.backend.httpwalletnotify.HTTPWalletNotifyHandler
param ip:Bound IP address. Default 127.0.0.1 (localhost).
parma port:Bound port. Default 28882.
Testing

To test that the wallet notifications are coming through

  1. Make sure cryptoassetshelper service is running
  2. Do curl --data "txid=foobar" http://localhost:28882 on the server where bitcoind is running
  3. You should see in the logs of cryptoassetshelper: Error communicating with bitcoind API call gettransaction: Invalid or non-wallet transaction id

Named UNIX pipe for bitcoind

Named pipes can be used to send incoming transactions notifications from locally installed cryptoassets daemons, namely bitcoind likes.

Cryptoassets helper service creates a name unix pipe.

Bitcoind or similar writes the transaction id to this pipe when updates are available for the transaction.

Named pipes are little bit more flexible than running a shell command, as you can do in-process handling of incoming transactions in a background thread, making it suitable for unit testing and such.

Example configuration:

# Locally running bitcoind in testnet
coins:
    btc:
        backend:
            class: cryptoassets.core.backend.bitcoind.Bitcoind
            url: http://foo:[email protected]:8332/
            walletnotify:
                class: cryptoassets.core.backend.pipewalletnotify.PipedWalletNotifyHandler
                fname: /tmp/cryptoassets-walletnotify-pipe

And corresponding bitcoind.conf:

walletnotify=echo $1 >> /tmp/cryptoassets--walletnotify-pipe

Please note that you might want to use timeout (gtimeout on OSX) to prevent bad behavior in the case cryptoassets helper service is unable to read from the time. Thus, incoming transaction notifications are discarded after certain timeout:

walletnotify=gtimeout --kill-after=10 5 /bin/bash -c "echo $1 >> /tmp/cryptoassets-unittest-walletnotify-pipe"
Options
param class:Always cryptoassets.core.backend.pipewalletnotify.PipedWalletNotifyHandler
param fname:Filename where the pipe is opened. Please note that any existing filename which same name is removed.
param mode:Unix file mode for the created pipe.

Redis pubsub for bitcoind

Use Redis pubsub for walletnotify notifications.

Redis offers mechanism called pubsub for channeled communication which can be used e.g. for interprocess communications.

  1. Connects to a Redis database over authenticated conneciton
  2. Opens a pubsub connection to a specific channel
  3. bitcoind walletnofify writes notifies to this channel using redis-cli command line tool
  4. This thread reads pubsub channel, triggers the service logic on upcoming notify

Example walletnotify line:

walletnotify=redis-cli publish bitcoind_walletnotify_pubsub %self

To install Redis on bitcoind server:

apt-get install redis-server redis-tools

Warning

The Redis authenticated connection is not encrypted. Use VPN or SSH tunnel to connect Redis over Internet.

Options
param class:Always cryptoassets.core.backend.rediswalletnotify.RedisWalletNotifyHandler
param host:IP/domain where Redis is running, default is localhost.
param port:TCP/IP port Redis is listetning to
param db:Redis database number
param username:optional username
param password:optional password
param channel:Name of Redis pubsub channel where we write transaction txids, default bitcoind_walletnotify_pubsub

block.io

Block.Io API backend.

Supports Bitcoin, Dogecoin and Litecoin on block.io API.

The backend configuration takes following parameters.

param class:Always cryptoassets.core.backend.blockio.BlockIo
param api_key:block.io API key
param password:block.io password
param network:one of btc, btctest, doge, dogetest, see chain.so for full list
param walletnotify:
 Configuration of wallet notify service set up for incoming transactions. You must use cryptoassets.core.backend.blockiowebhook.BlockIoWebhookNotifyHandler or cryptoassets.core.backend.blockiowebocket.BlockIoWebsocketNotifyHandler as walletnotify for incoming transactions for now. See below for more details.

Example configuration for block.io backend using websockets.

---
# Cryptoassets.core configuration for running block.io unit tests

database:
  url: sqlite:////tmp/cryptoassts-unittest-blockio.sqlite

coins:
    doge:
        backend:
            class: cryptoassets.core.backend.blockio.BlockIo
            api_key: yyy
            pin: xxxx
            network: dogetest
            walletnotify:
                class: cryptoassets.core.backend.blockiowebsocket.BlockIoWebsocketNotifyHandler

Wallet notifications over websockets

Handle notifications for incoming transactions using block.io websockets API.

https://block.io/docs/notifications

This will spin off a thread opening a websocket connection to block.io and listening for incoming events.

We use websocket-client library for websockets communications from Python.

Options
param class:Always cryptoassets.core.backend.blockiowebsocket.BlockIoWebsocketNotifyHandler

Wallet notifications over web hooks (HTTP)

Handle notifications for incoming transactions using block.io HTTPO POST webhook API.

https://block.io/docs/notifications

This will spin off a HTTP server in a separate IP and port to listen to HTTP requests made by the block.io. You need to specify an external URL how block.io can reach the public IP address of your server.

Options
param class:Always cryptoassets.core.backend.blockiowebhook.BlockIoWebhookNotifyHandler
param url:To which public URL your webhook handler is mapped. The URL must not be guessable and must cointain random string, so that malicious actors cannot spoof incoming transaction requests.
param ip:Bound IP address. Default 127.0.0.1 (localhost).
parma port:Bound port. Default 33233.
Securing the webhooks

Do not expose webhook service port directly to the internet. Instead, use your web server to create a reverse proxy behind a hidden URL, so you can safely receive notifications from block.io.

HTTPS support through Nginx

Here is an example Nginx web server configuration how decode HTTPS and then forward block.io requets to the upstream server running in the cryptoassets helper service process:

# Secret block.io webhook endpoint
location /blockio-account-nofity/xyz {
    proxy_set_header   Host $host;
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Host $server_name;
    proxy_set_header   X-Forwarded-Proto $scheme;
    proxy_pass http://localhost:33233;
}

blockchain.info

blockchain.info backend.

The support is disabled in this version

null

Non-functional cryptocurrency backend doing nothing. Use for testing purposes.

The backend configuration takes following parameters.

param class:Always cryptoassets.core.backend.null.DummyCoinBackend