In construction, not finished!!

Before hand notes...

Here I will outline the some of the elements to work with...

Source code

Source code for example this is here as a zip file. You can also connect to the kenai project, the svn for this sample will be here.

How to...

The video of how to build and test this project is here.

Technical Complexity

Simple

Purpose

• Monitor a stream. In this exercise, we are keeping 3 levels of data:
  1. The last day of data
  2. The last 3 entries for a specific key
  3. The last entry for a calculated value

Situation

• In the stock market, The last transactions provide valuable information to day trading. This model updates the internal db table, so that only the last stock transaction per market is kept in the table. This minimizes the table size and providing a higher throughput. To see the sample on how to send the data to an external db, please see here .
• Basic Stream calculations

Design

IEP Constructs Used

Process Flow

1. The events are acquired through the input.
2. From there, a one day table is kept in the inbound section of the process.
3. Using the last rolling 24 hours, an aggregation is done on the data to obtain the count of transactions and the sum of the money transferred for a symbol/market.
4. From that table, we keep only the last value for the symbol/market doublet key.
5. This result is kept into the table output.

Elements

• Stream Input
• Partitioned Window
• Relation Aggregator
• Relational Updates
• Table Output

Stream Input

Here we define the data, as we will consume it.

Relation Aggregator

This is the SQL statement used to aggregate the data we need.

Partitioned Window

This is the window we keep for presenting the derived event to our client layer. As you can see, we're keeping the same aggregation/partitioning (symbol/market) as in the aggregation operator.

We can see in this operator that we've set to keep the last 3 elements of the stream, for each stock. So this way, we have a shallow history on each element important for us.

Table Output

In this operator, we define how the data will be presented to the outside world, in a "known" table. Making it global allow us to keep the data even if the iep process is being undeployed.

Example Limitations

•  Unknown