FortiFly is advancing the science of real-time workload modeling, delivering insight into how people and systems interact. In this blog, we're introducing the latest version of our AI-enabled workload modeling capability: the Automatic Widget Identification Component, or AWIC.

Introducing AWIC

AWIC is FortiFly's AI-enabled capability that identifies, locates, and tracks mission-critical interface elements in real time. When combined with gaze-tracking hardware and AI-enabled head tracking, AWIC determines not only where an operator is looking, but what they're looking at—and why it matters in the context of a mission scenario.

Through AWIC and the information it collects,  we are then enabled to capture visual attention and determine the associated cognitive burden through real-time workload assessments as operators manage increasingly complex systems.

Tracking Visual Workload Where It Matters Most

In the example pictured above, a new pilot is learning how to operate an aircraft and, with that, how to shift their attention continuously between cockpit displays and a kneeboard tablet. Each interface features novel, highly specific icons that standard computer vision models can't reliably detect.

AWIC solves this by rapidly learning new interface elements—even in 3D environments—without requiring extensive retraining. As aircraft systems are updated or mission tools change, AWIC adapts right along with them. Trainers and mission designers gain confidence that workload monitoring reflects the current environment, not a version from months or years past.

Expanding into XR for Fully Immersive Scenarios

AWIC now integrates seamlessly with leading XR systems, such as the Varjo XR-4, and identifies and tracks interface components in both simulated and physical environments. This capability enhances visual workload estimation during immersive training scenarios—whether an operator is in a high-fidelity simulator or working hands-on with mission hardware.

Reducing Noise, Increasing Trust in Workload Insights

In high-tempo environments, precision matters. Our newest release (version 2.2) significantly improves the real-time detection of visual and cognitive workload by reducing false positives during AWIC's detections. Imperfect lighting, unexpected motion, and interface clutter are common in real-world settings—AWIC has been designed to perform reliably through it all.

Through recent advancements found in AWIC v2.2 we can:

• Improve detection accuracy

• Eliminate noise in workload assessment

• Provide clearer insights into operator focus

• Support safer, more adaptive system behaviors

The result is a stronger foundation for workload-informed automation, identifying when workload is off-nominal (ie., high workload or low workload) and then informing the system(s) so the system can then intelligently assist operators before attention becomes overloaded (or also to ensure underload situations do not occur).

Integration Anywhere Workload Matters

Best of all, AWIC's improvements reflect a broader mission: making real-time workload modeling accessible across industries. AWIC can be integrated into any third-party application, giving the FortiFly suite the flexibility to support:

• Aviation and defense operations

• Space and launch systems

• Advanced air mobility

• Remote supervision of fleets and robots

• Industrial training and safety-critical environments

Anywhere operators interact with complex systems, AWIC ensures their workload is understood and supported.

What's Next

FortiFly continues to push toward truly adaptive human–system teaming—where automation responds to real human needs in the moment. With AWIC's latest enhancements, that future is closer than ever.

Stay tuned as we continue expanding AWIC's capabilities and real-time workload modeling across the FortiFly platform.