Analytics and ChatGPT
Introduction
A chatbot that can write like a human, pass law and medical exams, and create solid code has taken the world by storm. It is almost impossible to avoid ChatGPT these days as the technology has taken over news articles, LinkedIn posts, watercooler chats, and even family conversations. The fastest-growing consumer application in history, it had more than 100 million monthly active users in January 2023.
So, what can ChatGPT do for the audit and certification industry?
How chatGPT works
Before we dive in, let’s look at how ChatGPT works to understand better the outputs it provides. ChatGPT is a Large Language Model (LLM) or statistical tool used to predict language without comprehending it. Here’s where it gets tricky: the chatbot produces “statistically plausible” answers that may sound perfectly correct but could be factually wrong. This happens when it combines inaccurate snippets of information, a phenomenon known as the hallucination of LLMs. Staying mindful of the output and maintaining a certain level of mistrust is critical to avoiding this common chatbot challenge.
Nevertheless, the technology behind ChatGPT can greatly improve your productivity if used correctly. Like any other tool, we must learn how it works, understand the best use cases for its purpose, and develop our technical skills to maximize ChatGPT’s potential. This is especially true as we have never worked with the new statistical, algorithm-based tool in such depth. We, as humans, are not good at thinking in statistical terms. We like black and white, 0 and 1, and trust information that makes sense to us. When working with such an LLM, we must develop a more critical, statistical way of thinking. ChatGPT answers that look black or white are, in fact, something different: statistically plausible answers stitched together from the information on the internet. When a model is trained on vast amounts of information publicly available on the internet, there is a degree of uncertainty in its accuracy.
This is the challenge for using ChatGPT in a business environment: it has been trained on widely accessible information and general knowledge. It has yet to be trained for a specific industry nor with your company’s data. While it can’t analyze numeric data to find outliers, trends, correlations, etc., it can help you find the right tools and algorithms and write how-to manuals.
ChatGPT Prompt
What ChatGPT is very good at is giving you first drafts and hints. So, let’s see what ChatGPT itself has to say about how to use it in the audit and certification industry:
Prompt:
Write an article on how to use ChatGPT for the audit and certification industry:
ChatGPT:
The audit and certification industry plays a crucial role in ensuring organizations go with relevant standards and regulations. With the increasing adoption of digital technologies, the industry is also looking for innovative ways to improve the efficiency and accuracy of its processes. ChatGPT, a powerful language model, can help organizations in the audit and certification industry to achieve these goals.
One of the primary ways that ChatGPT can be used in the audit and certification industry is through the automation of tasks. ChatGPT can be trained on specific regulations and standards, such as ISO 9001 or ISO 27001, to identify compliance issues in documents and reports. This can help auditors to quickly identify areas of concern, reducing the time and effort required for manual review.
ChatGPT can also be used to assist with risk assessment and analysis. By analyzing data from multiple sources, ChatGPT can identify potential areas of risk and highlight key issues that require attention. This can help organizations to prioritize their resources and focus on areas that are most likely to pose a threat to their operations.
Another way that ChatGPT can be used in the audit and certification industry is through its natural language processing capabilities. ChatGPT can be used to analyze customer feedback, support tickets, and other forms of unstructured data to identify common issues and areas for improvement. This can help organizations to improve their processes and services, leading to better customer satisfaction and retention.
Finally, ChatGPT can be used to provide real-time assistance to auditors and certification professionals. By integrating ChatGPT into existing systems, auditors and certification professionals can access real-time guidance and support on complex issues. This can help to improve the accuracy and consistency of audit and certification processes, leading to more reliable results.
Chat GPT Use Cases
Use case one:
Identify compliance issues in videos, audio, documents and reports
One potential use for ChatGPT in the audit and certification world is to identify compliance issues in documents, reports, video- and audio files from the auditee, auditor, or certification body. ChatGPT’s natural language processing (NLP) capabilities can analyze text, picture, video, and audio in documents and reports to identify specific terms and phrases commonly used in the standard, such as HACCP, continuous improvement, and recalls. It can then assess whether the terms have been appropriately addressed to identify potential compliance issues and flag them for review by a quality manager, auditor, reviewer, or integrity manager. ChatGPT can be trained on all relevant requirements and help identify areas of overlap and potential conflicts
Advantages
- Reduced time and effort required for manual review
- Improved accuracy and consistency of the compliance assessment process
- Especially beneficial for audits or organizations that must comply with multiple regulations and standards
- Helps identify areas of overlap and potential conflicts
Challenges
- No ability to analyze numeric/categorical evaluations common in checklists
- Needs a large data set of thousands of documents for each specific requirement to train
Opportunities
- Intact already has this use case built into its Intact Analytics tool.
- Build on Intact’s Analytics platform and NLP algorithm with already hundreds of thousands of documents
Use cases two and three:
Risk assessment and customer feedback
Use cases two and three may sound different, but they are two sides of the same coin – analyzing risk, which is why we are analyzing both cases together. In terms of customer feedback, the risk is churn or unhappy clients. In classical risk assessment, the risk is on the topic, e.g., food safety.
ChatGPT can assist with risk assessment and analysis by analyzing text or other media from multiple sources, including internal and external documents, news articles, meeting recordings, and social media. These files can consist of customer feedback, whistleblower information, auditor or reviewer reports, auditee documents and information about weather or soil conditions, social media sentiment analysis on a specific topic, or incidents reported in the news.
After retraining ChatGPT with specific industry documents and media describing risk factors like unsuited Critical control points (CCPs), lack of training, reputational risks, etc., the LLM can scan the mentioned media and flag passages connected to these risk factors. ChatGPT will continue to learn and improve as risk patterns and trends in data are identified to be used as input risk factors. This enables all audit and certification industry stakeholders to analyze, identify and prioritize potential risks and focus on areas that are prone to risk. Organizations can make more informed decisions and manage risk exposure exponentially more effectively and efficiently.
Advantages
- Once set up, more effective and efficient risk management
- Scans and flags large amounts of data
- Enables informed risk-based auditing
Challenges
- No analysis of numeric/categorical evaluations like common in checklists
- Needs large data set to train
- Algorithm needs data and resource-intense training and continuous retraining on specific industry and business operations
Opportunities
- Intact already has this use case built into its Intact Analytics tool.
- Build on Intact’s Analytics platform and NLP algorithm with already hundreds of thousands of documents
Use case four:
Real-time assistance to auditors and certification professionals
Usecase description
- We’ve broken this use case down into sub-cases to differentiate in more detail.
Guidance on how to interpret and score specific requirements
- ChatGPT can answer common questions an auditor or other stakeholders may have during an audit or certification process. It can guide the interpretation of requirements in a specific context and provide sample scorings and examples from other auditors or the standard setter to ensure that audits are accurate and consistent.
Real-time text, picture, video, and audio reviews with risk detection
- By integrating ChatGPT into the audit platform, it can analyze media in real time and provide suggestions for associated risks, improvements, or corrections. Based on these insights, the auditor or certification professional can get additional tailor-made checklists that go into detail on the risk-associated areas and/or skip irrelevant sections based on the context. This enables robust, informed, risk-based auditing and can help ensure more consistent audits while reducing the need for manual review.
Identifying compliance issues
- Similar to the 1.2 use case, ChatGPT can go beyond risk detection and provide analysis on compliance in general. Analyzing media with non-conformity descriptions in real time can address compliance with one or more standards accordingly.
Advantages
- Increases the accuracy, efficiency, and consistency of audit
- Points to potential risks in documents, non-conformity descriptions, root causes, and mitigations, etc.
Challenges
- No analysis of numeric/categorical evaluations like common in checklists
- Needs large data set to train
- Algorithm needs data and resource-intense training and continuous retraining on specific industry and business operations
Opportunities
- Intact already has this use case built into its Intact Analytics tool.
- Build on Intact’s Analytics platform and NLP algorithm with already hundreds of thousands of documents
Conclusion
No matter the use case, it’s important to remember that ChatGPT is still in a very early stage, and we need more time to truly understand the capabilities of this technology and the possible challenges and opportunities.