Fragment Selector Providers

Data.Path identifies a file. Data.Selector can identify a fragment inside that file. Data.SelectorFormat tells ProcessCore which selector language to use when comparing two fragments.

CSV Fragment Selector

The built-in CSV provider understands RFC 7111 row, column, and cell selectors.

With the provider, we can read and write textual selectors into their typed representation.

let provider = CsvFragmentSelectorProvider()

let columnSelector = provider.TryParse "col=1-3"

Some (ColumnSelector [{ First = Index 1
                        Last = Index 3 }])

let cellSelector = provider.TryParse "cell=2,2"

Some (CellSelector [{ Rows = { First = Index 2
                               Last = Index 2 }
                      Columns = { First = Index 2
                                  Last = Index 2 } }])

The provider can also relate two selectors. The relation can be either Exact, Contains, or Disjunct. In this case, col=1-3 contains cell=2,2.

// same selector is exact match
provider.Relate (columnSelector.Value) (columnSelector.Value)

Exact

// column contains cell
provider.Relate (columnSelector.Value) (cellSelector.Value)

Contains

// disjoint columns
provider.Relate (columnSelector.Value) ((provider.TryParse "col=4-6").Value)

Disjoint

Providers matter when both data nodes have selectors. Without a registered provider, ProcessCore can see that the paths match but cannot know whether col=1-3 contains cell=2,2.

let dataset = Dataset("fragment-demo")

let exportedColumns = Data(path = "measurements.csv", selector = "col=1-3", selectorFormat = CsvFragmentSelectorProvider.SelectorFormatUri, encodingFormat = "text/csv")
let measuredCell = Data(path = "measurements.csv", selector = "cell=2,2", selectorFormat = CsvFragmentSelectorProvider.SelectorFormatUri, encodingFormat = "text/csv")
let interpretedSample = Material("Interpreted sample", additionalType = "Sample")

let export = LabProcess("Export CSV")
export.AddOutputData(exportedColumns)

let interpret = LabProcess("Interpret selected cell")
interpret.AddInputData(measuredCell)
interpret.AddOutputMaterial(interpretedSample)

dataset.AddProcess(export)
dataset.AddProcess(interpret)

let beforeRegistration =
    exportedColumns.DownstreamMaterials(scope = dataset.AllProcesses())
    |> Seq.map (fun m -> m.Name)

beforeRegistration

seq []

Register the provider on the dataset. Registration is stored on the root dataset, so child datasets share the same selector-provider lookup.

dataset.RegisterFragmentSelectorProvider(provider)
// dataset.RegisterFragmentSelectorProvider(provider :> IFragmentSelectorProvider) // also works when upcast to interface

let registeredProvider =
    dataset.TryGetFragmentSelectorProvider(CsvFragmentSelectorProvider.SelectorFormatUri)
    |> Option.map (fun p -> p.SelectorFormat)

registeredProvider

Some "https://datatracker.ietf.org/doc/html/rfc7111"

let afterRegistration =
    exportedColumns.DownstreamMaterials(scope = dataset.AllProcesses())
    |> Seq.map (fun m -> m.Name)

afterRegistration

seq ["Interpreted sample"]

Custom Fragment Selector Providers

The idea behind the inclusion of generic fragment selectors syntax into the ProcessCore is so that any kind of fragment can be defined given a proper fragment selector specification. In the datamodel, this corresponds to an implementation of the IFragmentSelectorProvider interface, which can be registered on a dataset and will be used to relate any two selectors with the same SelectorFormat.

Usually, you should inherit from FragmentSelectorProviderBase<'Selector>, which implements IFragmentSelectorProvider and requires parsers and typed comparers. The provider parses strings into a typed selector and returns a semantic relation.

type PrefixSelectorProvider() =
    inherit FragmentSelectorProviderBase<string>()

    override _.SelectorFormat = "urn:example:prefix-selector"

    override _.TryParse(text: string) =
        if System.String.IsNullOrWhiteSpace text then None
        else Some (text.Trim('/'))

    override _.ToSelectorString(selector: string) =
        selector

    override _.Relate(container: string) (candidate: string) =
        if container = candidate then Exact
        elif candidate.StartsWith(container + "/") then Contains
        else Unknown

let customProviderResult =
    let p = PrefixSelectorProvider()
    (p :> IFragmentSelectorProvider).TryRelate "assay/table" "assay/table/row/1"

customProviderResult

Some Contains

What To Use When