Historical Market Data Management Systems
 | |  | |  | |  | |  |
Historical Market Data Management Systems stores real-time market data into
EyeDB; an Object Oriented Database Management System (OODBMS) based on the
ODMG 3 specification.
An arbitrary number of fields can be associated with equity throughout the equity's life time.
Fields such as the bid, ask, last, high, low, market center, liabilities, beta, pe, dividend, are
examples of such fields.
The Historical Market Data database can efficiently support and an arbitrary number of
these equity fields. At this time
Through the course of a day it is not the maximum number fields as described in items
one through eight that streams to the historical market data database; stocks type entities for
example typically stream bid, ask, last, trade time, and quote status to the database in no
particular order. Trade of a stock there is typically six fields that are streamed in no
particular order to the database. The database efficiently and dynamically adjust at an
time to meet the number of fields being sent.
The server can be feed with the "tic-by-tic" data created by the Ticker Plant
Server. That is the same data typically used by the Replay Server to repeat transmission of
market data for a day or set of days over any set of hours.
Although we intend to have the real-time storage issues go away by the end of 2009, at this
time the historical market data server is not capable of storing the entire market in real-time
--taking just the trades alone from just NYSE, AMEX, and NASDAQ produces on a average
600 trades per seconds--. In theory, it can be setup to delay store of the "tic-by-tic" market
data (something to if there is a demand for it)
NOTE: All the real-time databases I aware of, costing tens of thousands of dollars, executes
on powerful multi-processor systems keeping all transaction in memory until the end of the
trading day. Also they keep track of selected fields to product bars in a manner similar to
the types produce by example 5 of the C++ interface tutorials.
In a few day from, 2009/01/18, I am guessing 2009/01/27, I will publish the performance of
a bar type optimized Historical Marker Data database. The will determine what type of
performance increase if any is needed in the EyeDB product for this type of real-time data
insert.
The power in a database is how capable it is in managing complex data. The Historical
Market Data Management System more than meet the challenge for market data.
About EyeDB Project
It is my intension to become a regular contributor to the EyeDB project, especially given
the fact that the Historical Market Data database is build using EyeDB. Within that context
all questions regarding queries about OQL using this product will be filtered through EyeDB
general forum.
EyeDB provides an advanced object model (inheritance, collections, arrays, methods,
triggers, constraints, reflexivity), an object definition language based on ODMG ODL, an
object query and manipulation language based on ODMG OQL and programming
interfaces for C++ and Java.
Examples of Object Query Language (OQL) can be found on the EyeDB web pages.
EyeDB; an Object Oriented Database Management System (OODBMS) based on the
ODMG 3 specification.
An arbitrary number of fields can be associated with equity throughout the equity's life time.
Fields such as the bid, ask, last, high, low, market center, liabilities, beta, pe, dividend, are
examples of such fields.
The Historical Market Data database can efficiently support and an arbitrary number of
these equity fields. At this time
- Stocks has sixty nine fields defined
- Bonds has twenty seven fields defined
- Futures has thirty nine fields defined
- Futures Options has thirty six fields defined.
- Spreads has twenty one fields defined.
- Indexes has eighteen fields defined
- Calculated Indicators has ten fields defined.
- OPRA ---TBD but the database will dynamically compensate for any know data
type.
Through the course of a day it is not the maximum number fields as described in items
one through eight that streams to the historical market data database; stocks type entities for
example typically stream bid, ask, last, trade time, and quote status to the database in no
particular order. Trade of a stock there is typically six fields that are streamed in no
particular order to the database. The database efficiently and dynamically adjust at an
time to meet the number of fields being sent.
The server can be feed with the "tic-by-tic" data created by the Ticker Plant
Server. That is the same data typically used by the Replay Server to repeat transmission of
market data for a day or set of days over any set of hours.
Although we intend to have the real-time storage issues go away by the end of 2009, at this
time the historical market data server is not capable of storing the entire market in real-time
--taking just the trades alone from just NYSE, AMEX, and NASDAQ produces on a average
600 trades per seconds--. In theory, it can be setup to delay store of the "tic-by-tic" market
data (something to if there is a demand for it)
NOTE: All the real-time databases I aware of, costing tens of thousands of dollars, executes
on powerful multi-processor systems keeping all transaction in memory until the end of the
trading day. Also they keep track of selected fields to product bars in a manner similar to
the types produce by example 5 of the C++ interface tutorials.
In a few day from, 2009/01/18, I am guessing 2009/01/27, I will publish the performance of
a bar type optimized Historical Marker Data database. The will determine what type of
performance increase if any is needed in the EyeDB product for this type of real-time data
insert.
The power in a database is how capable it is in managing complex data. The Historical
Market Data Management System more than meet the challenge for market data.
About EyeDB Project
It is my intension to become a regular contributor to the EyeDB project, especially given
the fact that the Historical Market Data database is build using EyeDB. Within that context
all questions regarding queries about OQL using this product will be filtered through EyeDB
general forum.
EyeDB provides an advanced object model (inheritance, collections, arrays, methods,
triggers, constraints, reflexivity), an object definition language based on ODMG ODL, an
object query and manipulation language based on ODMG OQL and programming
interfaces for C++ and Java.
Examples of Object Query Language (OQL) can be found on the EyeDB web pages.
 |
 |
 |