NIST could serve as a clearing house for reports and test results on generative AI systems, with the extent of required testing and reporting depending on the risk level of the system's output. The generative AI delivery process involves builders creating models, trainers customizing models and interfaces, and deployers delivering applications using the models, while users generate output through prompts, applications, or plugins.

1. Generative AI Delivery Process
NIST could serve as clearing house for reports and test results
Extent of Testing and Reporting required depends on Risk Level
Output
Consumers
Output
Testing and
Evaluation
Consumers
choose how
to use outputs
Builders create models
possibly with API and
plugin capabilities
Trainers customize
models and create API
and plugins interface
Deployers deliver apps
possibly using in-context
learning, APIs, or plugins
Users use prompts,
apps, and/or plugins
to generate output
General
Testing
and
Regulatory
Reporting
Application
Area
Testing and
Regulatory
Reporting
Deployment
Environment
Testing and
Regulatory
Reporting
Production
Environment
Testing and
Regulatory
Reporting
Deployed
Applications
Foundation
Models
Outputs
Fine
Tuned
Model
NIST
Data
Sources
Data
Curation
and
Regulatory
Reporting
Curators select
data to be used
in initial training