DataStructureInterface API Documentation

Package

velox.api.layer1.messages.indicators

Overview

DataStructureInterface is the primary interface used by Bookmap strategies to query and retrieve historical market data. It provides methods to access both standard market events (trades, depth, BBO) and custom generated events from strategy generators.

Key Concepts

Time Format

All timestamps are in nanoseconds since epoch
Intervals are specified in nanoseconds
Time ranges are inclusive of start time (t0) and exclusive of end time (t1): [t0, t1)

Event Types

Standard Events: Built-in market data events (trades, depth, BBO, etc.)
Custom Events: User-defined events generated by custom strategy generators

Data Retrieval Modes

Aggregated: Data summarized over intervals (more efficient for large time ranges)
Non-aggregated: Individual events in chronological order (detailed but slower)

Nested Classes

`DataStructureInterface.StandardEvents` (enum)

Enumeration of built-in market event types available for querying.

`DataStructureInterface.TreeResponseInterval` (class)

Container for aggregated data over a specific time interval. Contains a map of events organized by event type.

Methods Overview

The interface provides 6 overloaded get() methods divided into two categories:

Category	Methods	Purpose
Standard Events	2 methods	Retrieve built-in market data (trades, depth, BBO)
Custom Events	4 methods	Retrieve user-generated events from custom generators

Standard Events Methods

Method 1: Multiple Intervals (Aggregated)

ArrayList<TreeResponseInterval> get(
    long t0,                    // Start time (inclusive) in nanoseconds
    long intervalWidth,         // Duration of each interval in nanoseconds
    int intervalNumber,         // Number of intervals to retrieve
    String alias,               // Instrument alias (e.g., "ESZ5.CME@RITHMIC")
    StandardEvents[] interestedEvents  // Array of event types to retrieve
)

Purpose: Retrieve standard market events aggregated into multiple time intervals.

Returns:

List of TreeResponseInterval with size = intervalNumber + 1
Element 0: Aggregation from earliest available data to t0 (exclusive) - essentially a "snapshot"
Elements 1 to N: Aggregations for each subsequent interval

Time Range Covered: [t0, t0 + (intervalWidth Ãƒâ€” intervalNumber))

Use Case: Efficient retrieval of data over large time ranges (e.g., hourly bars for an entire trading session)

Example:

// Get 5-minute aggregated bars for the last hour
long currentTime = provider.getCurrentTime();
long fiveMinutes = 5 * 60 * 1_000_000_000L;  // 5 minutes in nanoseconds
long oneHourAgo = currentTime - (60 * 60 * 1_000_000_000L);

ArrayList<TreeResponseInterval> bars = dataStructureInterface.get(
    oneHourAgo,                               // t0: Start 1 hour ago
    fiveMinutes,                              // intervalWidth: 5-minute bars
    12,                                       // intervalNumber: 12 bars (60 min / 5 min)
    "ESZ5.CME@RITHMIC",                      // alias
    new StandardEvents[] { StandardEvents.TRADE, StandardEvents.DEPTH }
);

// bars.get(0) = Snapshot (all data before oneHourAgo)
// bars.get(1) = First 5-minute bar [oneHourAgo, oneHourAgo + 5min)
// bars.get(2) = Second 5-minute bar [oneHourAgo + 5min, oneHourAgo + 10min)
// ... and so on

Method 2: Single Aggregation from Beginning

TreeResponseInterval get(
    long t1,                    // End time (exclusive) in nanoseconds
    String alias,               // Instrument alias
    StandardEvents[] interestedEvents  // Array of event types to retrieve
)

Purpose: Retrieve a single aggregation of standard events from the earliest available data up to time t1.

Returns: Single TreeResponseInterval containing aggregation for [-Ã¢Ë†Å¾, t1]

Use Case: Get cumulative statistics from session start to a specific time

Example:

// Get all trades from session start until now
TreeResponseInterval allTrades = dataStructureInterface.get(
    provider.getCurrentTime(),                // t1: Current time
    "ESZ5.CME@RITHMIC",                      // alias
    new StandardEvents[] { StandardEvents.TRADE }
);

TradeAggregationEvent trades = (TradeAggregationEvent) 
    allTrades.events.get(StandardEvents.TRADE.toString());

Log.info("Total session volume: " + trades.volume);

Custom Events Methods

Method 3: Multiple Intervals for Custom Events (Aggregated)

List<TreeResponseInterval> get(
    Class<?> strategyClass,     // Class of strategy that owns the generator
    String generatorName,       // Name of the custom generator
    long t0,                    // Start time (inclusive) in nanoseconds
    long intervalWidth,         // Duration of each interval in nanoseconds
    int intervalNumber,         // Number of intervals to retrieve
    String alias,               // Instrument alias
    Class<?>[] customEvents     // Array of custom event value classes
)

Purpose: Retrieve custom-generated events aggregated into multiple intervals.

Important: All events must be from the same generator owned by the specified strategy class.

Returns: Similar to Method 1 - list with snapshot + N intervals

Use Case: Retrieve aggregated custom indicator values over time

Example:

// Retrieve custom VWAP indicator values in 1-minute intervals
List<TreeResponseInterval> vwapData = dataStructureInterface.get(
    MyStrategy.class,           // strategyClass
    "VWAPGenerator",            // generatorName
    sessionStartTime,           // t0
    60_000_000_000L,           // intervalWidth: 1 minute
    390,                        // intervalNumber: 390 minutes (6.5 hour RTH)
    "ESZ5.CME@RITHMIC",        // alias
    new Class<?>[] { VwapValue.class }  // customEvents
);

Method 4: Time Range for Custom Events (Non-aggregated)

List<Object> get(
    Class<?> strategyClass,     // Class of strategy that owns the generator
    String generatorName,       // Name of the custom generator
    long t0,                    // Start time (inclusive) in nanoseconds
    long t1,                    // End time (exclusive) in nanoseconds
    String alias                // Instrument alias
)
throws IllegalArgumentException

Purpose: Retrieve all individual custom events within a specific time range [t0, t1).

Returns: List of individual event objects sorted chronologically

Exception: Throws IllegalArgumentException if generator uses aggregated events

Use Case: Get detailed custom events for precise analysis

Example:

// Get all custom order block signals between two times
List<Object> orderBlocks = dataStructureInterface.get(
    OrderBlockStrategy.class,   // strategyClass
    "OrderBlockDetector",       // generatorName
    rthOpenTime,                // t0: 9:30 AM
    rthCloseTime,               // t1: 4:00 PM
    "ESZ5.CME@RITHMIC"         // alias
);

for (Object event : orderBlocks) {
    OrderBlockEvent ob = (OrderBlockEvent) event;
    Log.info("Order Block at: " + ob.getTime() + ", Price: " + ob.price);
}

Method 5: All Custom Events Up To Time (Non-aggregated)

List<? extends CustomGeneratedEvent> get(
    Class<?> strategyClass,     // Class of strategy that owns the generator
    String generatorName,       // Name of the custom generator
    long t1,                    // End time (exclusive) in nanoseconds
    String alias                // Instrument alias
)
throws IllegalArgumentException

Purpose: Retrieve all custom events from the beginning up to time t1.

Returns: List of events for time range [-Ã¢Ë†Å¾, t1) sorted chronologically

Exception: Throws IllegalArgumentException if generator uses aggregated events

Example:

// Get all custom sweep signals up to current time
List<? extends CustomGeneratedEvent> sweeps = dataStructureInterface.get(
    SweepStrategy.class,        // strategyClass
    "SweepDetector",           // generatorName
    provider.getCurrentTime(),  // t1: Now
    "ESZ5.CME@RITHMIC"         // alias
);

Method 6: Single Aggregation for Custom Events

TreeResponseInterval get(
    Class<?> strategyClass,     // Class of strategy that owns the generator
    String generatorName,       // Name of the custom generator
    long t1,                    // End time (exclusive) in nanoseconds
    String alias,               // Instrument alias
    Class<?>[] customEvents     // Array of custom event value classes
)

Purpose: Retrieve a single aggregation of custom events from beginning to t1.

Returns: Single TreeResponseInterval for range [-Ã¢Ë†Å¾, t1]

Use Case: Get cumulative custom statistics up to a point in time

Important Notes

The "Snapshot" Element (Element 0)

When using aggregated methods (Methods 1 and 3), the 0th element is special:

Contains aggregation of ALL data from earliest available time to t0 (exclusive)
Acts as a "snapshot" or baseline state
Useful for initializing data structures before processing intervals
Example: Populate order book state before processing incremental updates

Performance Considerations

Aggregated methods are faster for large time ranges
Non-aggregated methods provide full detail but are slower
Keep intervalNumber reasonable (< few thousand intervals)
For very large queries, consider multiple smaller requests

Time Precision

All times must be in nanoseconds since Unix epoch
Use provider.getCurrentTime() to get current time in correct format
Convert human-readable times using:
```
long nanos = milliseconds * 1_000_000L;
```

Multi-Generator Queries

Each get() call can only retrieve from one generator
For multiple generators, make multiple separate calls
Cannot mix events from different generators in a single request

Event Type Arrays

For standard events:

new StandardEvents[] { 
    StandardEvents.TRADE, 
    StandardEvents.DEPTH, 
    StandardEvents.BBO 
}

For custom events:

new Class<?>[] { 
    MyCustomValue.class, 
    AnotherCustomValue.class 
}

Complete Usage Example

@Layer1SimpleAttachable
@Layer1StrategyName("Historical Data Analysis")
@Layer1ApiVersion(Layer1ApiVersionValue.VERSION1)
public class HistoricalAnalysis implements CustomModuleAdapter, Layer1ApiDataAdapter {

    private DataStructureInterface dataStructureInterface;
    private Layer1ApiProvider provider;

    @Override
    public void onUserMessage(Object data) {
        if (data.getClass() == UserMessageLayersChainCreatedTargeted.class) {
            UserMessageLayersChainCreatedTargeted message = 
                (UserMessageLayersChainCreatedTargeted) data;
            
            if (message.targetClass == getClass()) {
                // Request DataStructureInterface
                provider.sendUserMessage(new Layer1ApiDataInterfaceRequestMessage(
                    dsi -> {
                        this.dataStructureInterface = dsi;
                        analyzeRTHSession();
                    }
                ));
            }
        }
    }

    private void analyzeRTHSession() {
        String alias = "ESZ5.CME@RITHMIC";
        
        // Calculate RTH times (9:30 AM to 4:00 PM ET)
        long rthOpen = calculateRTHOpen();
        long rthClose = calculateRTHClose();
        
        // Get 5-minute bars for entire RTH session
        long fiveMinutes = 5 * 60 * 1_000_000_000L;
        int numBars = (int)((rthClose - rthOpen) / fiveMinutes);
        
        ArrayList<TreeResponseInterval> bars = dataStructureInterface.get(
            rthOpen,
            fiveMinutes,
            numBars,
            alias,
            new StandardEvents[] { StandardEvents.TRADE }
        );
        
        // Process each bar
        double sessionHigh = Double.MIN_VALUE;
        double sessionLow = Double.MAX_VALUE;
        long totalVolume = 0;
        
        for (int i = 1; i <= numBars; i++) {
            TreeResponseInterval bar = bars.get(i);
            TradeAggregationEvent trades = (TradeAggregationEvent) 
                bar.events.get(StandardEvents.TRADE.toString());
            
            if (trades != null) {
                sessionHigh = Math.max(sessionHigh, trades.maxPrice);
                sessionLow = Math.min(sessionLow, trades.minPrice);
                totalVolume += trades.volume;
            }
        }
        
        Log.info(String.format(
            "RTH Session Analysis - High: %.2f, Low: %.2f, Volume: %d",
            sessionHigh, sessionLow, totalVolume
        ));
    }
    
    private long calculateRTHOpen() {
        // Implementation to calculate 9:30 AM ET in nanoseconds
        Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("America/New_York"));
        cal.set(Calendar.HOUR_OF_DAY, 9);
        cal.set(Calendar.MINUTE, 30);
        cal.set(Calendar.SECOND, 0);
        cal.set(Calendar.MILLISECOND, 0);
        return cal.getTimeInMillis() * 1_000_000L;
    }
    
    private long calculateRTHClose() {
        // Implementation to calculate 4:00 PM ET in nanoseconds
        Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("America/New_York"));
        cal.set(Calendar.HOUR_OF_DAY, 16);
        cal.set(Calendar.MINUTE, 0);
        cal.set(Calendar.SECOND, 0);
        cal.set(Calendar.MILLISECOND, 0);
        return cal.getTimeInMillis() * 1_000_000L;
    }
}

Quick Reference Table

Need	Method to Use	Aggregated?	Event Type
Multiple intervals of standard events	Method 1	Yes	Standard
All standard events up to time	Method 2	Yes	Standard
Multiple intervals of custom events	Method 3	Yes	Custom
Custom events in time range	Method 4	No	Custom
All custom events up to time	Method 5	No	Custom
Single aggregation of custom events	Method 6	Yes	Custom

Summary

The DataStructureInterface is your gateway to historical market data in Bookmap. Use:

Aggregated methods for efficient large-scale queries
Non-aggregated methods when you need every individual event
Standard events for market data (trades, depth, BBO)
Custom events for your own generated indicators and signals

Always remember that timestamps are in nanoseconds and Element 0 in aggregated results is your snapshot/baseline state.

Package​

Overview​

Key Concepts​

Time Format​

Event Types​

Data Retrieval Modes​

Nested Classes​

DataStructureInterface.StandardEvents (enum)​

DataStructureInterface.TreeResponseInterval (class)​

Methods Overview​

Standard Events Methods​

Method 1: Multiple Intervals (Aggregated)​

Method 2: Single Aggregation from Beginning​

Custom Events Methods​

Method 3: Multiple Intervals for Custom Events (Aggregated)​

Method 4: Time Range for Custom Events (Non-aggregated)​

Method 5: All Custom Events Up To Time (Non-aggregated)​

Method 6: Single Aggregation for Custom Events​

Important Notes​

The "Snapshot" Element (Element 0)​

Performance Considerations​

Time Precision​

Multi-Generator Queries​

Event Type Arrays​

Complete Usage Example​

Quick Reference Table​

Summary​

Package

Overview

Key Concepts

Time Format

Event Types

Data Retrieval Modes

Nested Classes

`DataStructureInterface.StandardEvents` (enum)

`DataStructureInterface.TreeResponseInterval` (class)

Methods Overview

Standard Events Methods

Method 1: Multiple Intervals (Aggregated)

Method 2: Single Aggregation from Beginning

Custom Events Methods

Method 3: Multiple Intervals for Custom Events (Aggregated)

Method 4: Time Range for Custom Events (Non-aggregated)

Method 5: All Custom Events Up To Time (Non-aggregated)

Method 6: Single Aggregation for Custom Events

Important Notes

The "Snapshot" Element (Element 0)

Performance Considerations

Time Precision

Multi-Generator Queries

Event Type Arrays

Complete Usage Example

Quick Reference Table

Summary