Autonetics – The Approach to Intelligent Automation Design (Part 4: Design Considerations)

if we want users to like our software we should design it to behave like a likeable person.”  Alan Cooper,  “Father of Visual Basic” and the inventor of design personas

In part 3 we reviewed some of the challenges that arise when humans interact with automation.  In this blog post we’ll take a look at how we might address them in the future with an increasingly intelligent “co-pilot.”

We previously touched upon trustability and discussed the consequences of what might happen with either too little or too much trust in an automated system.  Yet there’s an even more basic question to ask which is how would you define (and measure) trust itself?  Is it: faith, predictability, dependability, reliability, openness or competence?  We’ll set aside the definition for now and turn to a potential model of how “trust” might be influenced – see Figure 1.

Figure 1.  A trust development model (adapted from this source).

What the model shows is that even prior to our interaction with a machine or system, we have some predetermined factors that can influence our degree of trust or acceptance.  Dispositional trust largely results from biological and environmental influences such as our age, culture, gender and personality and it tends to be static and hence (somewhat) easy to plan for.  Initial learned is, of course, your past experience (maybe via other systems such as video games) or perhaps your knowledge of the system based upon its reputation.

Situational, by its name, indicates that it’s variable and has 2 basic contexts: external variability and internal variability.  External variability is generated from the type of system, its complexity and the task level of difficulty for which the system is being used.  The operator’s workload is another factor that can influence this variable in terms of the individual’s monitoring capability of the system.  This is but a limited review of the key influencers and I would recommend that anyone interested in further understand access the original paper.  I might state here that I find most I&O automation systems not being very helpful in this context.

Internal variability is dependent upon context and one of those elements is subject matter expertise – within the domain but not specifically related to the system currently in use.  People with higher degrees of proficiency or competence tend to rely upon automation less than novices (and this is also observed in the preferences that I have personally observed, for example, between the use of GUIs and APIs/CLIs).  Interestingly, your mood is another influencer in addition to the level of individual stress and degree of restfulness.

While there’s much more to the model, space prevents me from a larger discussion although I’d like to address one more element of it.  The graphic also depicts situational factors not related to trust.  Research may suggest that while trust is key, it may only act as a guide and not completely determine reliance or usage.  Examples of these factors include automation alternatives, decision-making time frames, etc.  Past experiments have shown that operators were more inclined to leverage automation when their decision time was reduced.  Overall, hopefully you have a better grasp of just how difficult it is to provide the “right” interface for all human-to-machine interactions.

So, what does a good, trustable system look like?  Again, there are many considerations but I’ll focus on two – the system architecture and it’s human-to-machine interface.  In Figure 2 I’ve included an adaption from a paper entitled Original Postublication/362683588_Contestable_AI_by_Design_Towards_a_Framework/links/62f8589a52130a3cd71cee19/Contestable-AI-by-Design-Towards-a-Framework.pdf">Contestable AI by Design: Towards a Framework.

Figure 2.  A contestable architecture.

While there are other proposed architectures, I chose this one because of the word contestability, which in this case, refers to incorporating accountability within the system and makes the system responsive to human intervention (including feedback) throughout its lifecycle.  My specific add to the diagram was at the beginning and based upon the likely scenario of what occurred with regards to the design of the AF-447 cockpit where the algorithm (or avionics computer) developer, system architect and interface designer seemingly did not work collaboratively on all of the system-oriented “ilities.”

In terms of interface design, Figure 3 provides just a few representations of what I feel the UI might need to incorporate.  Observing from left to right, the first two examples are somewhat the same in that they are providing insight into probabilities of correctness but through different visualization methodologies.

Figure 3. User interface considerations.

If you live near a coastal area subject to hurricanes or cyclones, the second example is a tracking map that is likely very familiar. What’s great about these is that they don’t just show one model but help you understand the variance in presenting the predictions from several different models. This is something that I’d like to see incorporated more in I&O tools such as those within the AIOps market. The final example on the right is that of an autonomous vehicle but the component that I thought added most to trustability was its situation awareness of the surroundings showing that potential obstacles were being taken into consideration.

So far, we looked at many of the issues and considerations in developing a better human-to-machine environment.  In our final blog post of the series I’ll provide a model of what an Autonetics-oriented approach should entail from a disciplinary perspective to enable a much more functional and safer collaborative human and machine experience.