Overview
F5 Workforce AI Security evaluates risk at two stages of every AI interaction. When users send prompts to AI services, Workforce AI Security intercepts the interaction. Workforce AI Security assesses risk on the input (the user's prompt) and on the output (the response from the AI service). Each assessment produces an overall risk level. You can use the risk level to inform policy decisions, trigger alerts, or block high-risk interactions.
The risk assessment framework analyzes several factors:
The risk level of the AI service
Harmful content or prompt injection attempts
The intent behind the request
The sensitivity of any data detected in the interaction.
With these risk scores, you can decide which AI services to allow. You can also decide which interactions need more review. You can protect sensitive organizational data from unauthorized exposure.
Risk levels
Workforce AI Security classifies each risk signal as one of three levels.
Level | Description |
Low | Minimal risk. Standard business operations fall in this level. |
Medium | Moderate risk. This level can require monitoring or review. |
High | Significant risk. This level can trigger policy actions. |
The overall risk level for an interaction is Low, Medium, or High. The Critical level applies only to individual risk signals, such as data sensitivity.
Input risk assessment
Workforce AI Security analyzes five key factors to find the risk level of a user's prompt.
Service risk
Service risk is the inherent risk level of the AI service that receives the prompt. Workforce AI Security keeps a risk level for each known AI service in the Public Service Catalog. The level is based on three factors:
Training of models on user data
Lack of content moderation
Risk level of the country where the company that runs the service is based.
A reviewer can also set the risk level of a service by hand. A reviewer does this when these factors don't show a known risk, for example poor data practices.
When your organization uses private models, service risk is always Low. The data stays within your organization's control and isn't exposed to external AI providers.
Safety risk
Safety risk is a composite score from two threat detection systems. These systems analyze the content and structure of user prompts. The first factor is harmful content detection (also called toxicity detection). Harmful content detection finds offensive, inappropriate, or harmful language in prompts. The second factor is prompt injection detection. Prompt injection detection finds attempts to manipulate or jailbreak the AI system with instructions built to bypass safety controls.
Safety risk is the higher of the harmful content level and the prompt injection level.
Intent risk
Intent risk is the risk level based on what users want to do with their AI requests. Workforce AI Security analyzes the intent behind each prompt. It classifies the intent into business domains such as Finance, Legal, Health, and Engineering. Each domain maps to a risk level. The level depends on how sensitive the information in that domain is. The level also depends on the effect that errors or data exposure have on the organization.
Workforce AI Security refines intent domain classifications and their risk levels. It uses emerging threats, regulatory changes, and customer feedback. Workforce AI Security manages the specific mappings internally. The mappings can change as risk calculations evolve.
Data sensitivity
Data sensitivity classifies any sensitive information detected in user prompts. Workforce AI Security scans prompts for several types of sensitive data. Workforce AI Security assigns a classification level based on the potential impact if that data is exposed or mishandled.
Classification | Description | Examples |
Public | Commonly available information with lower sensitivity | Published website URLs, public business contact information |
Internal | Internal business data not meant for public disclosure | IP addresses, vehicle registrations |
Confidential | Sensitive data that needs protection | Email addresses, driver's licenses, tax IDs, cryptocurrency addresses |
Critical | Highly sensitive data that needs strict controls | Social Security numbers (SSNs), credit cards, bank accounts, passport numbers, healthcare IDs |
Data exposure impact
Data exposure impact is the potential consequence of data being exposed to the AI service. The calculation uses three inputs: how sensitive the detected data is, whether the service trains on user data, and whether intent risk is High.
The highest sensitivity classification of data in the prompt sets the base impact.
Data sensitivity | Base impact |
Critical or Confidential | High |
Internal | Medium |
Public or None | Low |
Two factors then change the base impact. First, if the AI service trains on user data, the exposure impact increases. The service can retain the data and add it to its models. This modifier elevates Low impact to Medium and any Medium or High impact to High.
Second, if the intent risk is High, the exposure impact also increases. High-risk domains involve more sensitive contexts, where data exposure has greater consequences. This modifier elevates Low impact to Medium and Medium impact to High.
Output risk assessment
When AI services return responses, Workforce AI Security analyzes the output to identify risks in the content delivered to users.
Service risk
Service risk for output assessment is the same as for input assessment. It is the inherent risk level of the AI service that generated the response.
For more about service risk, see The Public Service Catalog.
Safety risk
Safety risk for outputs is a composite score from two content detection systems. The first factor is harmful content detection (toxicity). Harmful content detection finds offensive, inappropriate, or harmful language in the response from the AI service. The second factor is bias detection. Bias detection finds biased or discriminatory content in the response. Such content can violate organizational policies or create legal exposure.
The safety risk calculation uses the same logic as input safety risk. Safety risk is the higher of the harmful content level and the bias detection level.
Overall risk calculation
Workforce AI Security combines the individual risk factors into an overall risk level. The level shows the complete risk profile of the interaction. Input and output assessments use different methods.
Input overall risk
The input risk calculation has two steps. First, Workforce AI Security calculates intermediate values. Then Workforce AI Security combines the values with a risk matrix.
Calculate intermediate risk values
Before it applies the risk matrix, Workforce AI Security calculates two intermediate values.
Interaction risk is the highest value among three factors:
Service risk (the inherent risk of the AI service)
Safety risk (composite of toxicity and prompt injection detection)
Intent risk (the risk level of the business domain).
For example, if service risk is Low, safety risk is Medium, and intent risk is Low, interaction risk is Medium. Medium is the highest of the three values.
Data exposure impact starts from the data sensitivity classification. Two amplification factors then change it:
The highest sensitivity classification detected sets the base impact.
If the service trains on user data, the impact increases.
If intent risk is High, the impact increases again.
For example, if Confidential data is detected, the base impact is High. If the service also trains on user data, the data exposure impact remains High. Now suppose only Internal data is detected, so the base impact is Medium. The service doesn't train on user data, and intent risk is Low. The data exposure impact remains Medium.
Apply the risk matrix
Next, Workforce AI Security combines the intermediate values with this risk matrix to produce the overall input risk.
Data exposure: Low | Data exposure: Medium | Data exposure: High | |
Interaction risk: Low | Low | Low | Medium |
Interaction risk: Medium | Low | Medium | High |
Interaction risk: High | Medium | High | High |
To use the matrix, find the row for the interaction risk level. Then find the column for the level of data exposure impact. The cell where the row and column meet shows the overall level of input risk.
For example:
Interaction risk: Medium, data exposure impact: High → overall input risk: High
Interaction risk: Low, data exposure impact: Medium → overall input risk: Low
Interaction risk: High, data exposure impact: Low → overall input risk: Medium.
Output overall risk
Output risk assessment is simpler than input assessment because outputs don't include data exposure considerations. The calculation has a single step.
The overall output risk is the higher of two values. They are service risk (the inherent risk of the AI service) and safety risk (the composite of toxicity and bias detection in the response). For example, if service risk is Low and safety risk is Medium, the overall output risk is Medium.
Policy integration
You can use risk scores to set up automated policy actions. These actions protect your organization from risky AI interactions. Risk levels serve as triggers for policy rules in the Default Policy or Group Policies.
To set up policy actions based on risk levels, in GOVERN, go to Default Policy or Group Policies. In policy rules, you can specify actions to take when interactions meet certain risk thresholds. You can block the interaction, require approval, redact sensitive data, or log the interaction for review.
Currently, Workforce AI Security manages the risk calculation thresholds and the underlying risk factors, and you can't customize them. The ability to customize risk thresholds and calculations is part of the product roadmap. With that ability, you can match the thresholds and calculations to your organization's risk tolerance.
View risk scores
You can view risk assessments for individual interactions and analyze aggregated risk trends across your organization.
To view risk scores for individual user interactions, in OBSERVE, go to User Activity. The User Activity view shows each intercepted AI interaction, the calculated risk scores, detected data types, and any policy actions that triggered.
To view aggregated risk analytics and trends over time, in OBSERVE, go to Insights. The Insights dashboard shows risk patterns in visualizations. Use the visualizations to find high-risk users, services, or domains, and decide whether they need more policy controls or user training.
Examples
The following examples walk through the complete risk calculation. Each example shows how a different interaction scenario results in a different overall risk level.
Example: low-risk interaction
A user asks a low-risk AI service to summarize a public news article. The prompt contains no sensitive data and uses professional language appropriate for business use.
Individual risk factors
Service risk: Low
Safety risk: Low (no toxicity or injection detected)
Intent risk: Low (general-purpose request)
Data sensitivity: none detected.
Determine interaction risk
Service risk, safety risk, and intent risk are all Low, so interaction risk is Low.
Determine data exposure impact
The base impact from data sensitivity is Low, because no sensitive data is detected.
No modifiers apply, because the service doesn't train on user data and intent risk isn't High.
The data exposure impact is Low.
Apply the risk matrix
The risk matrix gives an overall input risk of Low for interaction risk Low and data exposure impact Low.
Example: elevated risk from sensitive data
A user asks a low-risk AI service to help draft internal documentation. The prompt includes employee email addresses for a distribution list.
Individual risk factors
Service risk: Low
Safety risk: Low (no toxicity or injection detected)
Intent risk: Low (general-purpose request)
Data sensitivity: Confidential (email addresses detected).
Determine interaction risk
Service risk, safety risk, and intent risk are all Low, so interaction risk is Low.
Determine data exposure impact
The base impact from data sensitivity is High, because the data classification is Confidential.
No modifiers apply, because the service doesn't train on user data and intent risk isn't High.
The data exposure impact is High.
Apply the risk matrix
The risk matrix gives an overall input risk of Medium for interaction risk Low and data exposure impact High.
A Medium result alerts you that confidential data was shared with an AI service. This result can trigger policies that require data redaction or additional review.
Example: high-risk interaction
A user sends a prompt that contains Social Security numbers (SSNs) to a public AI service that trains on user data. The prompt also includes language that triggers prompt injection detection. The language suggests an attempt to manipulate the AI system.
Individual risk factors
Service risk: Medium
Safety risk: High (prompt injection detected)
Intent risk: Low (general-purpose request)
Data sensitivity: Critical (SSN detected).
Determine interaction risk
Service risk is Medium, safety risk is High, and intent risk is Low, so interaction risk is High. High is the highest of the three values.
Determine data exposure impact
The base impact from data sensitivity is High, because the data classification is Critical.
The service trains on user data, so the High impact remains High.
The data exposure impact is High.
Apply the risk matrix
The risk matrix gives an overall input risk of High for interaction risk High and data exposure impact High.
A High result can trigger blocking policies. Blocking policies prevent the exposure of critical data to an external AI service. The policies also require immediate security review.