It has been shown that AI models in real-world contexts do not always perform as expected [based on pre-deployment] testing environments. Post deployment issues include … hallucination, sycophantic behavior, security exploits, and false claims … models have been found to detect when they are being evaluated … the variability introduced by AI models, coupled with the many system components … and user interactions, forms a large attack surface
–NIST, Trustworthy and Responsible AI
This month the National Institute of Standards and Technology (NIST) issued a report about the challenges of post-deployment testing of AI systems in organizations. Real-world testing has been a standard practice of any tech installation. But AI tools present new challenges.
The report confirms areas of concern NIST raised in its 2023 publications, but with M365 Copilot and other newly mainstream business-use AI, the attack surface has grown … [6 min. read]