One Rig Data Fabric: Connecting High-Frequency Edge Analytics to WITSML and Federated Learning
The first two articles in this series followed drilling data in opposite directions: up to the office through WITSML, ETP, and OSDU, and down to the control floor through Modbus, OPC UA, and high-frequency measurement. This one is about the layer where those two paths should meet in the field. A rig-side data fabric can preserve the decision value of fast measurements, run engineering and machine-learning applications locally, reconcile every rig source into one governed WITSML publication layer for the office, and support governed learning across a fleet, all without forcing every signal onto one rate or one protocol.
🔗 The missing integration between the rig and the office
The previous two WillCo articles followed two different paths through the drilling data architecture. From Rig Data to Drilling Autonomy traced structured well data from the rig to the office through WITSML, ETP, and OSDU. Beyond WITSML moved down to the control floor, where Modbus, OPC UA, PLCs, edge platforms, and high-frequency measurements support local monitoring and control. The natural next question is how those two paths should work together in the field, on a real rig, at the same time.
They should meet at an edge data fabric. That layer receives measurements at the rate each use case actually requires, aligns them with operational context, and makes them available to local engineering applications. The same layer can then publish selected measurements, calculated channels, alerts, and event evidence to an office-facing WITSML environment. The result is not one universal data rate for everything. It is one governed architecture that preserves high resolution where latency and transient behavior matter, and reduces or packages information where the office needs context rather than every sample.
This article proposes that architecture as WillCo engineering analysis. The individual protocols, data objects, modeling examples, and federated-learning concepts are grounded in the standards and publications listed in the references. The claim is not that any single product does all of this today. The claim is that these pieces belong together, and that the edge data fabric is the layer that connects them.
⏱️ One signal chain has several different clocks
As Beyond WITSML set out in detail, calling a drilling feed "real time" hides a chain of different clocks: the sensor samples, the controller scans, the protocol polls, the gateway acquires, the historian records with compression, and only then does an office service publish. The rate the office sees is set by the slowest stage in that chain, not by the fastest sensor on the rig.
The architecture should preserve the native or useful rate of each signal until the intended decision has been made. A vibration waveform, a standpipe-pressure transient, a hook-load trend, a survey, and a fluid-property report do not all require the same rate. Sending every channel at the highest available frequency wastes compute, storage, and bandwidth and makes quality control harder; reducing every channel to one common office rate discards patterns local applications need. The design rule is specific: acquire each signal fast enough to represent the phenomenon and the model that will use it, preserve the full useful resolution inside the edge boundary, and publish upward according to the information requirement rather than the raw sample rate.
✅ The edge should receive decision-grade data
High-frequency acquisition has value only when the samples can be trusted and combined. Modbus may expose a number at a register address without carrying its complete engineering meaning. An OPC UA server can provide richer metadata, but the value of that metadata depends on the information model the vendor implemented. Other sources may use WITS, WITSML, MQTT, proprietary APIs, or files produced by service systems. The edge layer has to reconcile all of these before an engineering model consumes them.
At minimum, the integration layer should establish the following before any calculation runs:
This is the real purpose of the edge data fabric. It converts a collection of asynchronous numbers into a time-aligned operational state that multiple applications can use without each rebuilding the same mappings independently. Without it, every application on the rig privately reinvents the same fragile translation, and every private translation is another place for meaning to be lost or applied wrong.
⚙️ Real-time engineering at the rig
Once the data has been synchronized and contextualized, the edge can support a class of engineering applications that is difficult to operate from an office feed alone. Automated torque-and-drag models can compare measured hook load and torque with modeled response while drilling or tripping. Hydraulic models can update frictional pressure losses, equivalent circulating density, surge and swab pressure, and hole-cleaning indicators as operating conditions change. Signal analytics can identify oscillations, pressure disturbances, equipment-state changes, and deviations between expected and measured behavior.
Published work supports the use of real-time data in automated torque-and-drag and hydraulic workflows. It also exposes the practical constraint: these models must merge measurements and contextual inputs that arrive with different frequencies, quality, and update behavior. A faster signal does not automatically make a model more accurate. It can make the result more responsive and preserve short-duration behavior, but reliability still depends on sensor quality, time alignment, calibration, model assumptions, and correct operational state.
The useful update rate also differs by calculation. A transient pressure diagnostic may need a faster input stream than a steady-state hydraulics calculation. A torque-and-drag model may benefit more from reliable block position, depth, hook load, pipe state, and friction calibration than from an indiscriminate increase in every channel rate. The edge architecture should therefore support multiple processing rates rather than forcing every application into one cycle. That is the whole reason to keep the fast data on the rig: the calculations that most reward high resolution are exactly the ones an office-facing feed configured for lower-rate publication may not serve.
Once the data is decision-grade, high-frequency rig-sensor signals, often in the 5 to 100 Hz range depending on the measurement, carry the oscillations, pressure transients, and short equipment-state changes that a 1 Hz office feed has already averaged away, so a torque-and-drag or hydraulics calculation run locally on the full-resolution signal can be more faithful than the same calculation run on a decimated office stream. Running it at the rig also removes the round trip: there is no need to ship raw data to the office, compute there, and stream results back to the floor, and a real-time decision on the rig cannot always absorb that added latency.
🧩 The models need data that no sensor sends
High-frequency sensor streams are only half of what a real-time model consumes. Torque and drag, hydraulics, ECD, and surge and swab all depend on static or slow-changing configuration data that no sensor emits continuously, and a model is never more correct than that configuration:
The current bottom-hole assembly: its component list, dimensions, weights, and stabilizer positions.
The wellbore geometry: each casing and liner string that has been run, its size and setting depth, and the open-hole section below the last string.
Fluid properties and rheology in use, updated as the mud is treated or changed.
The well plan and directional survey that define the trajectory the string follows.
A torque-and-drag model run without the correct BHA and casing program, or a hydraulics and ECD model run without the correct string geometry and mud rheology, will produce results that may be precise but materially wrong, no matter how fast the sensors sample beneath them.
This is why unreliable configuration is worse than missing configuration. A model that knows a value is absent can flag the gap, widen its uncertainty, or refuse to run; a model handed a stale BHA or the planned string in place of the as-built one gets no such warning and produces a confident number that is wrong without knowing it, which is the state operators trust most and should trust least. Earlier WillCo articles made this same point about rig state and machine-learning labels, and it applies with equal force to the configuration these engineering models depend on.
It is worth being precise about where the gap actually is, because it is easy to overstate. WITSML already defines objects for exactly this configuration: BhaRun and Tubular for the assembly, WellboreGeometry for the casing and hole record, FluidsReport for mud properties, and DrillReport for the daily record, with the BhaRun structure carried from WITSML 1.4.1.1 into 2.0. The standard is not missing.
What is inconsistent in practice is how completely those objects are populated, who owns keeping them current, whether they are updated at the moment a BHA is run or a string is set rather than end of day, and whether they are delivered to the edge at all. On many rigs the BHA and casing record still reach the floor as manual entries, late spreadsheets, or a line in a morning report, so the model at the edge is working from a structure that exists but is stale or absent. The weakness of running these systems at the edge is not the absence of a standard object; it is population, ownership, currency, and delivery.
The consequence for the architecture is direct. The edge data fabric has to treat configuration as a first-class input, governed like any sensor channel: a stable identity, an effective time such as when a BHA was run or pulled and when a casing string was set, a version, and a provenance tag. The edge must always know which BHA and which casing string are in effect at a given moment, and it must update the affected models when they change at a trip or a casing point. Missing or stale configuration should raise a data-quality fault, not pass silently into a calculation.
Closing that gap with what a rig has today, rather than waiting for the industry to standardize delivery, runs in two horizons. The near-term bridge is event-driven digital hand-off from the accountable source: the directional service or rig contractor populates the as-run BHA the moment it is picked up, and the mud engineer delivers each mud check digitally rather than only on paper, each hand-off stamped with effective time, version, and provenance and validated on arrival. The morning report has a place in this too, but a narrow one. It is a useful daily baseline and reconciliation source where its fields are complete and machine-readable, not a complete or real-time record, because it is daily and retrospective and will lag any change made mid-tour. It backstops the event-driven hand-offs; it does not replace them.
The target state retires the manual steps: the existing WITSML objects populated at the source and delivered continuously, with automated fluid-property measurement such as in-line rheometers feeding mud properties directly, so the edge holds current configuration without a person retyping it, and publishes one reconciled configuration record into the governed office layer alongside the sensor channels.
📦 Three data products should leave the edge
The edge produces more than a reduced copy of raw rig data. Its outputs fall into three different classes, and each class needs its own publication policy. Treating them as one stream is what pushes teams toward the wrong instinct of moving everything upstairs at full rate.
Table 1. Three classes of edge output and the publication pattern each one calls for. WillCo analysis, grounded in the WITSML and ETP data model (Energistics).
Data product | Examples | Publication pattern | Purpose |
Continuous derived channels | Calculated ECD, expected hook load, modeled torque, margins and scores | Regular WITSML or ETP channel updates at a useful office rate | Maintain operational awareness and trends |
Events and alerts | Model deviation, pack-off indicator, vibration severity, sensor-quality failure | Immediate event or channel update, with severity and model context | Notify the office when attention is required |
High-frequency evidence | Buffered samples before, during, and after a detected event | On-demand or policy-triggered transfer, not a permanent full-rate stream | Support diagnosis, replay, calibration, and model development |
A circular edge buffer makes the third class practical. The platform continuously retains a rolling high-frequency window. When an alert is triggered, the system preserves the interval before, during, and after the event. The office receives the derived result immediately and can request the associated raw evidence later, when investigation or model improvement justifies the transfer. The full-rate stream never has to run continuously to the office, yet the fast data is not lost when it finally matters.
🏢 WITSML becomes the office publication layer
WITSML should not be treated as the control bus for this architecture. The immediate control loop remains inside the tested rig-control environment. WITSML provides the office-facing structure for well data, while ETP can move selected channel data and notifications efficiently. The edge uses that path to publish an integrated view assembled from several local sources, not to close a loop.
In multi-provider operations, the office may receive separate WITSML feeds from the rig contractor, mud logger, directional and measurement-while-drilling service, and specialized systems. Those feeds can carry different time bases, depth references, channel names, and quality behavior, so the office is left to stitch together overlapping and sometimes conflicting versions of the same well. Wherever that fragmentation exists, it is the same fragmentation the edge already removed on the rig, rebuilt in town.
This is where the integration already done at the edge pays off a second time. The edge data fabric is the layer where every source is received and aligned, so it is also the natural place to publish from. High-frequency channels, the rig-contractor record, mud logging, directional and survey data, fluid reports, and third-party providers can enter the same layer, be reconciled against one time base, one asset identity, and one depth reference, and leave as a governed, reconciled office publication layer rather than several competing versions of the same well. The office consumes one coherent picture instead of stitching partial ones together, and every value in it was already made decision-grade at the point the sources were combined. This is WillCo architectural analysis rather than a description of how any single product ships today.
Reconciled does not mean flattened, and this only works if it is governed rather than merged blindly. The publication layer has to preserve each value's source identity, quality, ownership, and lineage, and it has to record the transformations applied along the way. When two systems report the same channel, an explicit conflict-resolution rule names a source of truth by domain, and entitlements govern who may see what. The layer also has to buffer, monitor its own health, and fail over when a source drops.
Consolidation at the edge is the default, not a mandate. Where a contract, a validation requirement, or an accountability line requires a service to publish its own native feed in parallel, that feed remains, and the reconciled layer references it rather than overwriting it.
Calculated outputs can be represented as derived channels when the implementation defines their meaning, units, source, and derivation. WITSML 2.x provides ChannelDerivation for exactly this; in older 1.4.1.1 environments the same calculated curves are exchanged through agreed mappings rather than a native derivation concept. Either way, the presence of a technical container does not create universal semantics for a proprietary model output. Operators, drilling contractors, and application vendors still need governed names, identifiers, units, quality rules, and version metadata.
The edge publication service should label every output as measured, calculated, estimated, predicted, or recommended, and carry the algorithm or model version, the calculation time, the input-quality status, and a confidence or validity indicator where those concepts apply. Without that distinction, an office user may read a machine-learning prediction as a physical measurement, or treat an uncalibrated estimate as a verified engineering result.
Alerts need the same discipline. Where the receiving system supports it, a notification can travel through an agreed WITSML mapping; otherwise an adjacent event service carries it while linking to the same well, time, depth, channel, and model identifiers used in WITSML. The objective is traceable integration, not forcing every message into a structure that does not express it cleanly.
🤖 Machine learning closes a second loop
The architecture creates a second flow above the operational data path. Many drilling models are first developed in the office from historical datasets that have already been filtered, averaged, or sampled at office-facing rates. Deploying such a model at the rig exposes it to higher-resolution signals and to local conditions that may be underrepresented in the original training data.
Each rig is different. Equipment response, sensor calibration, BHA design, formation behavior, drilling practices, and data availability vary across a fleet. It helps to keep four processes distinct, because they are often collapsed into the word "adapt." Inference is a deployed model producing outputs; it changes nothing about the model. Calibration tunes a small number of parameters to local conditions within an approved range. Retraining rebuilds the model from data and is a governed, deliberate step. Federated aggregation combines updates across rigs into a new global candidate. Deployment alone does not retrain a model: a model exposed to a rig's own high-frequency data does not silently rewrite itself, and any move from inference toward calibration or retraining is a controlled decision, not a side effect. The result does not have to be an independent model at every rig; a more manageable architecture combines a validated global model with a bounded local calibration or adaptation layer where the use case supports it.
Federated learning is one mechanism for improving the global model without continuously shipping all raw training data to a central store. Participating rigs train or adapt a common model locally and send controlled model updates, such as parameters or gradients, together with validation metrics. A central service aggregates the accepted contributions and produces a candidate global version. The original federated-averaging work describes exactly this pattern: learning a shared model by aggregating locally computed updates while the training data stays distributed.
🌐 Federated does not mean automatic
A drilling fleet is a difficult federated-learning environment because the local datasets are not identically distributed. One rig may drill a different formation, run a different control system, or operate with a different sensor package. The number and quality of training examples will also vary. Averaging every local update without qualification can degrade the global model rather than improve it.
The aggregation service should know which model version produced each update, which operating envelope generated the data, how much valid data supported the update, and how the local model performed against an approved validation set. Updates that fail quality, security, drift, or performance checks should not enter the global candidate. Secure aggregation and privacy controls may also be required, because model updates can reveal information about local data under some conditions. Keeping raw data on the rig reduces data movement, but it does not by itself guarantee privacy or integrity.
The candidate global model must then pass testing, evaluation, verification, and validation before deployment. NIST's AI Risk Management Framework treats model validation and system validation as lifecycle activities rather than one-time development tasks. In a drilling application, the release process should also include version control, engineering approval, staged rollout, performance monitoring, and rollback. Federated learning is a way to pool experience across rigs, not a license to push unreviewed models to a fleet.
🚧 Learning does not grant control authority
The control boundary from the previous article stays fixed, and the learning layer does not move it. Interlocks, limit enforcement, command arbitration, and fail-safe behavior belong in the validated control architecture, where they can be tested and held accountable. A model that predicts well is still a source of advice, not a controller.
🗺️ A practical deployment sequence
An operator does not need to rebuild the entire rig architecture at once. The integration can progress through controlled stages, each of which delivers value on its own and sets up the next.
🎯 The objective
The objective is not to send every sample everywhere. It is to preserve the decision value of rig data long enough to make the right local calculation, retain the evidence needed to understand important events, and publish a trustworthy operational picture to the office.
A well-designed edge data fabric lets the same measurements support several outcomes without forcing every consumer onto the same frequency or protocol. Local engineering applications receive the resolution they need. Office systems receive integrated measurements, derived channels, alerts, and evidence as one governed, reconciled publication layer rather than several competing feeds. Machine-learning systems learn from local conditions without requiring every raw dataset to leave the rig.
A final dimension this article has set aside on purpose is security. Moving a trained model from the office to the rig, and moving model updates back across the fleet through federated aggregation, both cross the boundary between the rig network and the company internal network. That boundary is not only a control-authority line, it is a trust and network-security line, and every data path and learning loop described here has to be designed to respect it. It also requires IT and OT considerations that depend on each company's own infrastructure, so there is no single blueprint. That subject is large enough to stand on its own, and it is one of the themes this series will take up in a different article.
Related WillCo analysis
From Rig Data to Drilling Autonomy: What WITS, WITSML, ETP and OSDU Actually Do - the office-facing data path this article publishes into.
Beyond WITSML: Turning High-Frequency Rig Data into Edge Decisions - the control-floor path and the control-authority boundary this article carries forward.
Why Pure Machine Learning Is Not Enough for Drilling Optimization - why intelligence cannot substitute for well-structured, high-resolution inputs.
The BHA Static-Data Timing Gap: Why Your Real-Time Models Are Running Blind - why stale or planned configuration in place of the as-built state leaves a model confidently wrong.
Reaming Is Not Washing: Getting Borehole-Enlargement Data Right - why a model trained on unreliable labels is more dangerous than no model at all.
📚 References and further reading
Data standards and transfer
Energistics (an affiliate of The Open Group), WITSML data standards and Energistics Transfer Protocol (ETP). Source for WITSML as the office-facing well-data standard and ETP as a transfer protocol supporting near-real-time streaming, notifications, and data-object operations. Publication rates and time or depth intervals are implementation-specific; the cited standards do not prescribe one universal surface rate.
Energistics, WITSML documentation, including the log-channel organization and the ChannelDerivation concept in WITSML 2.x. Source for representing calculated outputs as derived channels; in 1.4.1.1 environments equivalent curves are exchanged through agreed mappings. That a technical container exists does not create universal semantics for proprietary model outputs, which is WillCo analysis.
Energistics, WITSML 1.4.1.1 to 2.0 object mapping. Source for the existing WITSML configuration objects referenced here (BhaRun, Tubular, WellboreGeometry, FluidsReport, DrillReport) and for the BhaRun structure carried from 1.4.1.1 into 2.0. The characterization of the gap as population, ownership, currency, and delivery rather than a missing standard object is WillCo analysis.
Modbus Organization, Modbus specifications and implementation guides. Source for the register-based, non-self-describing data model referenced here.
OPC Foundation, OPC Unified Architecture (OPC UA). Source for the self-describing information model, quality and timestamp metadata, and companion-specification approach referenced here.
Real-time drilling engineering
P. Shahri et al., "Automated Real-Time Torque and Drag Analysis: A New Milestone Toward Drilling Optimization," SPE-191426-MS, 2018. Primary source for the use of real-time data in automated torque-and-drag workflows. See also the readable summary by C. Carpenter, Journal of Petroleum Technology, 2019.
E. Cayeux et al., "Automatic Drilling Fluids Monitoring," SPE/IADC-204041-MS, 2021. Source for automated, real-time monitoring of drilling-fluid and hydraulic behavior.
M. Karimi et al., "Real-Time Drilling Hydraulics Monitoring," AADE-22-FTCE-078, 2022. Source for real-time hydraulics, ECD, and surge-and-swab monitoring approaches.
J. Sugiura and S. Jones, "A Drill Bit and a Drilling Motor with Embedded High-Frequency (1600 Hz) Drilling Dynamics Sensors," SPE Drilling & Completion 34(3), 2019 (SPE-194138-PA). Source for the 1,600 Hz in-bit measurement quoted in the statistic box.
Machine learning and governance
H. B. McMahan et al., "Communication-Efficient Learning of Deep Networks from Decentralized Data," Proceedings of Machine Learning Research 54, 2017. Source for federated averaging: learning a shared model by aggregating locally computed updates while training data remains distributed. The non-identically-distributed-data difficulty is discussed in this and subsequent federated-learning literature; its application to a drilling fleet is WillCo analysis.
NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023. Source for treating model and system validation as lifecycle activities rather than one-time development tasks.
K. Stouffer et al., Guide to Operational Technology (OT) Security, NIST Special Publication 800-82 Revision 3, 2023. Source for the principle that operational-technology performance, reliability, and safety constraints govern where control authority may reside.
Standards and protocol facts are attributed to Energistics, the Modbus Organization, the OPC Foundation, and NIST, and to the named drilling and machine-learning publications, retrieved September 2026. The only field measurement quoted as a number is the 1,600 Hz in-bit rate from Sugiura and Jones (SPE-194138-PA, 2019). The integrated architecture, the two-loop framing, the control-authority boundary, the treatment of configuration data as a governed input, the governed office publication layer, and the deployment sequence are WillCo analysis. Figures are conceptual, and no field-performance or economic claims are made.
🔤 Acronyms
AI RMF | Artificial Intelligence Risk Management Framework (NIST) |
BHA | Bottom-hole assembly |
ECD | Equivalent circulating density |
ETP | Energistics Transfer Protocol |
Hz | Hertz (samples or cycles per second) |
MQTT | Message Queuing Telemetry Transport |
NIST | National Institute of Standards and Technology |
OPC UA | OPC Unified Architecture |
OSDU | Open Subsurface Data Universe |
OT | Operational technology |
PLC | Programmable logic controller |
WITS | Wellsite Information Transfer Specification |
WITSML | Wellsite Information Transfer Standard Markup Language |
Reuse and citation. This article may be reproduced in whole or in part, for any purpose, provided WillCo Drilling Consulting is credited as the source and a link to willcodrilling.com is included.
Suggested citation: WillCo Drilling Consulting, "One Rig Data Fabric: Connecting High-Frequency Edge Analytics to WITSML and Federated Learning," willcodrilling.com, September 21, 2026.
Comments