Instrumenting a forever stream using hs-opentelemetry

In the context of hs-opentelemetry, instrumenting an action involves creating a span at the beginning of the action and ending the span after the action.

Example,

function :: IO ()
function = do
  ...
  s <- startSpan
  action
  endSpan s
  ...

Consider the following code and requirements:

function :: IO ()
function =
    Stream.repeat processBatch
      & Stream.fold Fold.drain
  where
    processBatch = do
      ...

Requirements:

1. We want to have visibility on processBatch.
2. We want to have visibility over the entire stream execution as well.

Instrumenting processBatch is straightforward. We just wrap it around a span.

But instrumenting the entire stream is not. We can add the startSpan at the beginning using Stream.before but where will you add endSpan. You might say Stream.after but this is a stream that runs forever and there is no after forever.

A nicer approach that gives more visibility for such streams is to group the stream and instrument the groups.

Forever Stream:

[0] -> [1] -> [2] -> ... -> [999] -> [1000] -> ...

Grouped Forever Stream (groupsOf 100):

[ [0] -> [1] -> ...[99] ] -> [ [100] -> ... -> [199] ] -> ...

Since the number of elements in this group is finite in nature, instrumenting these groups is simpler. We startSpan at the beginning of every group and endSpan at the end of every group.

We can do something like this:

function :: IO ()
function =
    Stream.repeat processBatch
      & Stream.groupsOf 100 wrapSpan
      & Stream.fold Fold.drain
  where
    processBatch = do
      ...

This way we get the live instrumentation data frequently.