The purpose & features of the Notification API¶

Automated notifications are widely used in software systems to keep stakeholders informed. Specific events may require notifying relevant parties in advance to ensure they are aware and able to respond before subsequent events take place. This approach is particularly useful for sequencing operations in pipelines, as identifying issues earlier can save resources compared to detecting them later in the process.

Please refer to the following example payload in the sections that follow. This payload has been submitted to the Notification API in a hypothetical scenario in which a failure has occurred in a LIMS system trying to accession a sample identified by 40STDY246690 upon sample manifest upload.

Info

The failure is due to invalid data for the Country field in a sample manifest, where the country name United Kingdom has been incorrectly entered as United Kindm. The ID of the sample manifest has been included as the aggregator_id field.

{
  "channels": [
    {
      "type": "SLACK",
      "recipient": [
        "X_LAB_STAKEHOLDERS"
      ],
      "subject": "Accessioning Failure: Invalid input data for manifest",
      "content_type": "text",
      "template_id": "X_ACCESSIONING_INVALID_INPUTS",
      "fields": {
        "attributes": {
          "key": "Country",
          "value": "United Kindm"
        },
        "sample-id": "40STDY246690"
      }
    }
  ],
  "priority": "BATCH",
  "aggregator_id": "21784312"
}

Note

The details regarding the payloads are discussed in detail in other sections of the user guide.

Features of the Notification API¶

In essence, the Notification API delivers notifications to a given destination channel. Built on top of that are the following features that allow the users to customise the content and the behaviour of each notification.

Priority-based Notification Dispatch¶

The Notification API allows the user to dispatch messages based on three priorities. The priority is identified for each notification with the priority key.

NORMAL: This is the default priority that is assigned to any notification that is received by the Notification API.
HIGH: A high priority notification is a notification that is required to be delivered with minimum delay.
BATCH: Notifications of BATCH priority are notifications that are likely to be grouped up (i.e., "aggregated") into a single notification.

Infrastructure Configuration

The underlying infrastructure configuration have different wait times for each priority. A BATCH priority is configured with a relatively longer wait time for to let the notifications aggregate. The consumers do not have a handle to update the wait time. A HIGH priority notification would get immediately dispatched to downstream systems which parse, synthesise and dispatch the notification(s) to the channels listed in the request.

Key-based Message Aggregation¶

Some use cases require aggregating multiple notifications into a single message, allowing the Notification API to deliver a consolidated notification to the appropriate stakeholders. The Notification API allows this using a discriminator aggregator_id.

For example, in the scenario above, if multiple notification arrive to the Notification API having the same aggregator_id - which could be the ID of the sample manifest, and the multiple notifications could be for the samples within the manifest - the underlying system would aggregate them within a time window configured by the C4E (Centre for Enablement), and send the aggregated message as a batch.

Aggregation Criteria

The aggregation criteria is as follows:

In a case where multiple messages \({M_1, M_2, ... , M_m, ... , M_n}\) (where \(n \in \mathbb{N}\)) arrive at the aggregator with any \(m\) \((m \in [1, n])\) messages having the same aggregation ID \(A\), same recipient \(R\), same template ID \(T\), same subject \(S\), and for same channel \(C\), the \(m\) messages would be aggregated into a single notification.

Warning

For the aggregation to happen, priority must be BATCH.

Template-based Message Synthesis¶

The Notification API enables message generation to be delegated to the notification system using template engines. The client does not have to prepare large messages - which eventually leads to larger payloads for invoking the Notification API - for the notification process. The client merely has to pre-populate different templates into the Integration Hub through the C4E and send along the payload with the necessary arguments for the Notification API to generate the message.

Arguments for the template

In the payload above, the arguments are given via the fields key. If there are any issues related to populating, the Notification API would flag it as a failure.

Notification-delivery Status Check¶

The Notification API allows checking the status of a notification identified by a notification ID. Using this API, the client would be able to determine if the notification succeeded or failed.

Notification Failures

Validation failures — such as invalid message bodies, non-existent recipients, or missing templates — are checked synchronously when the Notification API receives a request. These failures result in an immediate error response to the client. If the request passes validation but the notification fails to be delivered, clients can use the polling API to check the delivery status of a specific notification. Request tracing along the internal component is available for the C4E to troubleshoot if required.

Identifying the usage for synchronous processes¶

Assume that a step \(S_i\) in a work pipeline invoked the subsequent step, or a sub-step within \(S_i\), and it errored. Because the process is synchronous, the calling process gets immediately notified after the next step (or the sub-step) returns. Ideally, the caller would then have to have properly-written up client code in order to invoke certain notification APIs like Slack. The client code should include wrappers for error handling, observability, retry logic, rate limiting, and logging, as well as managing authentication and configuration for each channel.

Note that this also applies not only for erroring scenarios, but also for scenarios where the subsequent calls are successful, but it is required to notify a broader user group (e.g., a group of lab users).

The Integration Hub Notification API has many built-in capabilities to mitigate the issues discussed. The ipaas-framework, which is the core for Integration Hub integrations, encapsulates error handlers, retriers, different types of loggers as well as observability tools provided by the PaaS (Platform-as-a-Service).

Identifying the usage for asynchronous processes¶

Assume that a step in a work pipeline invoked an asynchronous job \(J\) that triggered a certain step in \(S_i\), and \(S_i\) failed due to an issue \(E_i\).

Because the nature of \(J\) is asynchronous, if there are no notification systems in play, \(E_i\) may go undetected until a step further downstream in the pipeline, or it might go unnoticed.

A Notification API would help in this case in order to send a certain notification (an "alert" in this case) to pre-defined stakeholders notifying of \(E_i\). \(E_i\) could be notified with the pre-defined stakeholder groups with a proper content layout, which is also pre-defined.