Onboarding a new Price Feed using Configuration

This document will walk you through configuring and deploying a price feed oracle in IntelliX that fetches data from multiple exchanges, each with its own response format, and publishes the aggregated result to multiple blockchain networks.

Firstly, let's create a intellix.yaml to store the configuration. We begin by defining the oracle name and unique id type in the configuration file

name: BTC/USDT
id: 5d34c60b-e665-44c3-a55e-34851a359dc9

sources:
   ...

processors:
   ...

publishers:
   ...

validators:
   ...   
   
output_schema:
   ...

The configuration contains the following main sections:

Sources: This section defines how data is retrieved from external sources. It will include the URLs of the data sources (e.g., Binance, Coinbase, Kraken, Bitfinex) from where the BTC/USDT price is fetched.
Processors: This section will define how the raw data fetched from sources is processed. Here, we will use JSONPath processors to extract the relevant price data from the API responses of the exchanges.
Validators: This section specifies the validation method to be applied to the fetched and processed data. In this example, a price validator that combines both the median and average of the prices from different exchanges is used to ensure accuracy before publishing.
Publishers: This section contains details about where the validated price data will be published. The contract addresses for different blockchain networks (e.g., Bitlayer, BSC, Merlin) are included to ensure the data is delivered to the correct destination for on-chain use.
Output Schema:The schema defines the structure of the final output, which in this case is represented by the BTC/USDT price.

Sources Configuration

Define the Data Sources and Processing Rules

Each exchange provides its price feed in JSON format, but the structure of the response varies from one exchange to another. For example, Binance may return a price directly in a simple JSON object, while Kraken or Bitfinex might embed the price deeper within a nested JSON structure.

To handle these differences, IntelliX uses JSONPath expressions to specify exactly how the price data should be extracted from each source. For example:

Binance

URL: https://api.binance.com/api/v3/ticker/price?symbol=BTCUSDT

Sample Response:

{
  "symbol": "BTCUSDT",
  "price": "43500.23000000"
}

JSONPath: $.price

Coinbase

URL: https://api.exchange.coinbase.com/products/BTC-USD/ticker

Sample Response:

{
  "trade_id": 4729088,
  "price": "43500.23",
  "size": "0.01000000",
  "bid": "43500.22",
  "ask": "43500.23",
  "volume": "120.00000000",
  "time": "2023-10-01T12:34:56.789Z"
}

JSONPath: $.price

Kraken

URL: https://api.kraken.com/0/public/Ticker?pair=XBTUSD

Sample Response:

{
  "error": [],
  "result": {
    "XXBTZUSD": {
      "a": ["43501.00000", "1", "1.000"],
      "b": ["43500.00000", "1", "1.000"],
      "c": ["43500.23000", "0.10000000"],
      "v": ["1200.12345678", "3200.98765432"],
      "p": ["43400.12345", "43350.98765"],
      "t": [1000, 2000],
      "l": ["43000.00000", "42900.00000"],
      "h": ["44000.00000", "44500.00000"],
      "o": "43200.00000"
    }
  }
}

JSONPath: $.result.XXBTZUSD.c[0] (for last price)

Bitfinex

URL: https://api-pub.bitfinex.com/v2/ticker/tBTCUSD

Sample Response:

[
  43500,    // BID
  1,        // BID SIZE
  43501,    // ASK
  1,        // ASK SIZE
  43500.5,  // DAILY CHANGE
  500.5,    // DAILY CHANGE PERC
  43500.23, // LAST PRICE
  1200,     // VOLUME
  44000,    // HIGH
  43000     // LOW
]

JSONPath: $[6] (index 6 contains the last price)

These URLs and corresponding JSONPaths will allow you to fetch real-time BTC/USDT price data from each exchange and integrate them into your IntelliX oracle setup.

The configuration will look like the following:

sources:
    
    - id: binance
      type: url
      url: "https://api.binance.com/api/v3/ticker/price?symbol=BTCUSDT"
      
    - id: coinbase
      type: url
      url: "https://api.exchange.coinbase.com/products/BTC-USD/ticker"
    
    - id: kraken
      type: url
      url: "https://api.kraken.com/0/public/Ticker?pair=XBTUSD"
    
    - id: bitfinex
      type: url
      url: "https://api-pub.bitfinex.com/v2/ticker/tBTCUSD"

update_frequency: 60  # Fetches data every 60 seconds

processors:
    - type: json_path
      source: [binance, coinbase]
      paths: 
        price: "$.price"
        ts: HTTP_DATE_HEADER
      
    - type: json_path
      source: kraken
      path: "$.result.XXBTZUSD.c[0]"
    
    - type: json_path
      source: bitfinex
      path: "$[6]"

Mode of Operation

IntelliX allows you to configure the mode of operation for your oracle. You can choose between the following modes:

push: The oracle will periodically fetch and publish data based on the defined update frequency.
pull: The oracle only responds when a request is made to pull data.
push_and_pull: The oracle operates in both modes, periodically pushing data and also responding to pull requests.

When the mode is set to push, you must also define the update frequency and, optionally, an end_time. The update frequency determines how often the oracle will fetch and publish data, while the end time specifies the time (in UTC or epoch) the oracle should stop pushing data. If an end time is not provided, the oracle will stop pushing data after the gas has run out.

mode: "push"  # Options: push, pull, push_and_pull
update_frequency: 60  # Required if mode is 'push'; fetches data every 60 seconds
end_time: "2025-01-01T13:00:00Z" # When data pushing should stop

When the push or push_and_pull mode is employed, the configuration also requires a publishers section, listing the target contract addresses where data will be periodically published.

In contrary, when pull mode is used, the data is fetched and processed only when requested by a smart contract. The smart contract directly queries the oracle to retrieve the necessary data at the time of execution. Since there is no need to automatically push or publish the data on-chain at regular intervals, the publishers section becomes unnecessary in this scenario.

More information about the pull contract requirements is available here.

In the current version, only pull mode is supported

Publishers Configuration (optional)

When the push or push_and_pull mode is employed feeds can be automatically published to multiple networks.

IntelliX requires a standard contract to be deployed on each target network before publishing any data from an oracle. You will need to provide contract addresses for each target network. For example, if we want to publish our price feeds to BitLayer, BSC and Merlin Chain, the configuration will look like the following:

publisher:
    - id: bitlayer
      contract_address: "0x1234567890abcdef1234567890abcdef12345678"
    - id: bsc
      contract_address: "0xabcdefabcdefabcdefabcdefabcdefabcdefabcdef"
    - id: merlin
      contract_address: "0xabcdef123456abcdef123456abcdef123456abcdef

More information about the push contract requirements is available here.

Validators Configuration

The validator section in this configuration specifies how the fetched data should be validated before being published on-chain. In this case, the validator type is set to price_avg_median, which indicates that the data from multiple sources (like exchanges in a price feed use case) will undergo the validation process described here.

validator:
   type: price_avg_median

Output Schema Configuration

The output schema in the IntelliX configuration defines the structure and format of the data that is ultimately produced by the oracle after processing. This schema ensures that the data, once validated, adheres to a predefined format that is compatible with smart contracts and can be easily consumed by blockchain applications.

In this specific case, the output schema is represnted by a single floating point value.

output_schema:
  type: "ufixed256x18"  # ABI-compatible type for a floating point

Full Configuration Example (Pull)

Below is a complete configuration file that specifies multiple data sources, network-specific contract addresses, and JSONPaths for each exchange.

name: BTC/USDT
id: 5d34c60b-e665-44c3-a55e-34851a359dc9
mode: pull
sources:
    - id: binance
      type: url
      url: "https://api.binance.com/api/v3/ticker/price?symbol=BTCUSDT"
    - id: coinbase
      type: url
      url: "https://api.exchange.coinbase.com/products/BTC-USD/ticker"
    - id: kraken
      type: url
      url: "https://api.kraken.com/0/public/Ticker?pair=XBTUSD"
    - id: bitfinex
      type: url
      url: "https://api-pub.bitfinex.com/v2/ticker/tBTCUSD"
processors:
    - type: json_path
      source: [binance, coinbase]
      path: "$.price"
    - type: json_path
      source: kraken
      path: "$.result.XXBTZUSD.c[0]"
    - type: json_path
      source: bitfinex
      path: "$[6]"

validators:
   type: price_avg_median
   
output_schema:
  type: "ufixed256x18"  # ABI-compatible type for a floating point

Deployment

With the configuration complete, we can proceed to deploy the oracle using the IntelliX SDK. Before deployment, ensure that the account has sufficient gas to cover the transaction costs for publishing data.

const { IntelliX } = require('intellix-sdk');
const fs = require('fs');
const yaml = require('yaml');
const path = require('path');

// Initialize IntelliX SDK with your credentials
const intelliX = new IntelliX({
    key: ...
});

// Load the YAML configuration file
const configFilePath = path.join(__dirname, 'price-feed-config.yaml');
const configFile = fs.readFileSync(configFilePath, 'utf8');
const config = yaml.parse(configFile);

// Deploy the oracle with the loaded configuration
async function deployOracle() {
    try {
        const response = await intelliX.deploy(config);
        console.log('Oracle deployed successfully:', response);
    } catch (error) {
        console.error('Error deploying oracle:', error);
    }
}

// Call the function to deploy the oracle
deployOracle();

PreviousArchitecture Components NextPush and Pull Flows and Contracts

Last updated 1 year ago