Event Store, Backfill and Replay
Overview of Mediation Event Store
The Event Store is an intermediate data storage solution in the mediation system that serves as both a source to ingest data and a target to store event data. It allows you to capture, store, and process event data for various purposes including:
- Billing: Aggregate usage data across different systems before processing it for invoicing
- Analytics: Store event data for business intelligence and usage pattern analysis
- Reporting: Maintain historical data for compliance and operational reporting
- Data consolidation: Bring together data from disparate systems before further processing.
Using the Event Store
Using the Event Store is particularly valuable in the following scenarios:
Business need | Using Event Store |
---|---|
Multi-region data collection | When you need to collect data from different geographic regions that may generate data at different times, the Event Store can serve as a central repository before processing the aggregated data. |
Time-shifted processing |
When you need to ingest data at one time but process it at a later point (e.g., collecting usage throughout the month but only processing at month-end) |
Data transformation |
When data needs to be collected in different formats for subsequent processing, the Event Store can serve as the intermediary storage. |
Error recovery and reprocessing | If there are issues with downstream processing, having data in the Event Store allows for data reprocessing without needing to re-extract from source systems. |
Data validation | When you need to verify data completeness or correctness before submitting it for billing or other critical processes. |
Use Cases
Use Case Example | Scenario | How Event Store helps |
---|---|---|
Multi- region business consolidation |
A global company has operations in Asia, Europe, and North America. Each region has its own billing system that generates usage data. The company needs to consolidate all this data before processing it for customer invoicing. |
The Event Store collects data from all regions as it becomes available throughout the month, then a single meter pulls from the Event Store to process the consolidated data once all regions have reported. |
Usage-based Pricing with Verification |
A SaaS provider charges customers based on API usage. They want to collect raw usage data throughout the month, but need to verify and clean the data before calculating charges. |
They use the Event Store to collect the raw usage events, then at month-end, run a verification process before determining the final billable amount. |
Regulatory Reporting with Historical Access | A telecom company must maintain records of all customer communications for regulatory purposes. | They store all communication events in the Event Store, which allows them to generate monthly compliance reports while maintaining the ability to query data during the billing cycle, before pushing that data into a data warehouse. |
Meter with Event Store as a source
To create a meter with Event Store as the source, follow the instructions below:
- Create a Custom Meter. See Create a new meter using custom method.
- From Sources, select Event Store.
- In the Event Store settings page, select an existing Event Store or click Create New Event Store
- To Create new Event Store:
- Enter the Event Store Name.
- Select the Event Definition. The selected event definition must have the event time field.
- Click Save.
- To Create new Event Store:
Meter with Event Store as a target
To create a meter with Event Store as the source, follow the instructions below:
- Select the Event Store as the source.
- In Event Store settings page, select an existing Event Store or Create New Event Store
- Map the event fields for event store event definition. The
startDateTime
- Click Save.
Meter with Event Store replay
To create a meter with Event Store as the source, follow the instructions below to replay:
- Click the Run button in the meter list page or in the meter page.
- Select one of the following Event Store Data Scope:
- All-time: If you want to load all the data from the selected even store
- Specify time frame: Toload the data within the time range from the event store. Select the Start Date and End Date.
- Click Run.
Meter with Event Store backfill
To create a meter with Event Store as the source, follow the instructions below to backfill:
- Click to enter the event store details page. If the event store is in progress of writing data, the backfill button will be grayed out.
- Click the Backfill button and select delete events.
- Download the template and upload the event to be deleted. Limit: CSV file with max 1000 rows.
- Confirm the delete action.
- When the delete process finishes, you will see a delete success message on the page.
- The event store status will also be updated from
in use and writing
toin use and idle
.
Notification
- Selecting the same event stores as multiple different output targets in the same meter is not permissible.
- Selecting the same event stores as source and target in the same meter is not allowed.
- When the event store status is
writing data
ordeleting data
, the delete event function under backfill will be disabled until the event store status is back toorphan
orIdle
status.