Testing and CI/CD for Trusted AI

Testing and CI/CD are software engineering practices that aim to ensure the quality, reliability, and security of software applications. They are especially important for AI applications, which involve complex and dynamic data, models, and algorithms.

What is Software 1.0 and Software 2.0?

Software 1.0 refers to the traditional way of developing software by writing code that specifies the logic and rules of the application. Software 2.0 refers to the emerging way of developing software by using machine learning (ML) models that learn from data and generate code or behavior.

Why do we need testing and CI/CD for AI?

AI applications pose unique challenges and risks for testing and CI/CD, such as:

• Data quality and availability: AI applications depend on large and diverse datasets that may be noisy, incomplete, biased, or outdated.
• Model performance and robustness: AI models may have high accuracy on average, but low accuracy on edge cases or adversarial inputs. They may also degrade over time or become obsolete due to data drift or concept drift.
• Ethical and social implications: AI applications may have unintended or harmful consequences on human values, rights, and well-being, such as privacy, fairness, accountability, transparency, and explainability.

Testing and CI/CD can help address these challenges and risks by:

• Automating and streamlining the data, model, and code workflows of AI development
• Detecting and preventing errors, bugs, vulnerabilities, and anomalies in AI applications
• Ensuring that AI applications meet the functional, non-functional, and ethical requirements of the stakeholders
• Enabling continuous improvement and innovation of AI applications

How do we implement testing and CI/CD for AI?

Testing and CI/CD for AI involve applying software engineering best practices to the data, model, and code components of AI applications. Some examples are:

• Data testing: Validating the quality, consistency, completeness, relevance, and diversity of the data used for training and testing AI models
• Model testing: Evaluating the accuracy, robustness, efficiency, scalability, interpretability, and fairness of AI models on various metrics and scenarios
• Code testing: Verifying the functionality, usability, security, reliability, maintainability, and portability of the code that implements or integrates AI models

Real World Machine Learning “Pipelines”

Real World Machine Learning “Data Engine”

How Tesla’s Data Engine works

• The FSD computer in each Tesla car records and sends any inaccuracies or discrepancies between its actions and the human driver’s actions to Tesla’s servers.
• Tesla’s servers use these inaccuracies to create unit tests, which are scenarios that the FSD neural network should be able to handle correctly.
• Tesla’s servers also use these inaccuracies to search for similar situations in the vast amount of data collected from all the cars in the fleet and create a well-labeled dataset.
• Tesla’s servers use this dataset to re-train the FSD neural network using machine learning algorithms and improve its performance and functionality.
• Tesla’s servers deploy the updated FSD neural network to the cars’ FSD computers in “shadow mode” to compare its actions with the human driver’s actions.

Why Tesla’s Data Engine matters

• Tesla’s Data Engine allows them to leverage the power of big data analytics and artificial intelligence in their self-driving cars.
• Tesla’s Data Engine enables them to address the long tail problem of autonomous driving, which is the challenge of handling rare or complex situations on the road.
• Tesla’s Data Engine ensures that the FSD neural network is constantly improving and not regressing or introducing new errors.

What is “Testing” for AI vs Traditional Software Testing

Why do we need testing for ML systems?

ML systems pose unique challenges and risks for testing, such as:

• Data quality and availability: ML systems depend on large and diverse datasets that may be noisy, incomplete, biased, or outdated.
• Model performance and robustness: ML models may have high accuracy on average, but low accuracy on edge cases or adversarial inputs. They may also degrade over time or become obsolete due to data drift or concept drift.
• Ethical and social implications: ML systems may have unintended or harmful consequences on human values, rights, and well-being, such as privacy, fairness, accountability, transparency, and explainability.

Testing ML systems can help address these challenges and risks by:

• Detecting and preventing errors, bugs, vulnerabilities, and anomalies in ML systems
• Ensuring that ML systems meet the functional, non-functional, and ethical requirements of the stakeholders
• Enabling continuous improvement and innovation of ML systems

How do we test ML systems?

Testing ML systems involves applying software engineering best practices to the data, model, and code components of ML systems. Some examples are:

• Data testing: Validating the quality, consistency, completeness, relevance, and diversity of the data used for training and testing ML models
• Model testing: Evaluating the accuracy, robustness, efficiency, scalability, interpretability, and fairness of ML models on various metrics and scenarios
• Code testing: Verifying the functionality, usability, security, reliability, maintainability, and portability of the code that implements or integrates ML models

What are some tools and techniques for testing ML systems?

There are different tools and techniques that can help us test ML systems effectively and efficiently. Some examples are:

• Data validation tools: Tools that help us check the schema, statistics, anomalies, drifts, and distributions of our data. For example: TensorFlow Data Validation, Great Expectations, Deequ.
• Model validation tools: Tools that help us measure the performance of our models on various metrics and scenarios. For example: TensorFlow Model Analysis, Scikit-Learn, PyTorch.
• Code validation tools: Tools that help us check the syntax, style, coverage, complexity, and security of our code. For example: PyLint, PyTest, Bandit.

Testing “Software 1.0” vs “Software 2.0”

“Beyond Accuracy: Behavioral Testing of NLP Models with CheckList”

Examples of “real world” testing

Examples of “real world” testing as a starting point