Turning AI into potential gain

Creating quality: How AI supports item development

Let first consider perception of quality and rigor of distractors

Over the years of both creating and working with test item back creators I have shared in the struggles of creating high-quality test items. It takes a long time and once you have the items, it goes through lots of discussion and arguing by the quality assurance team on the validity, level of difficulty and choice of distractors in MCQs. AI generated items are no different, they still need to go through the same quality assurance process, except the time to write, make changes and adjust levels of difficulty are greatly reduced.

There should always be human intervention in the process before any final version is released. Items need to be checked to ensure they are pedagogically sound for the test level and subject.

AI is consistently good and very quick at creating unbiased questions with strong distractors, probably better than a human when dealing with large amount of test items required. Ask most SMEs writing questions and they will be the first to agree that AI is a big help, gives them more to work with and allows more time for review.

Building trust in regulated environments

Secondly, let's consider trust and credibility for regulations and compliance

Awarding bodies and Professional Institutes are trusted institutions and many are regulated and have sets of rules that must be strictly adhered to. It is important that all content created be unbiased, without error, and trusted by the public. Any use of AI must therefore comply with the regulated environment, i.e. separated from the public Internet and held secure and available for external verification. Using an AI generator like iQ keeps all your data secure, and private. It is separate from the public Internet and all responses are auditable. iQ ensures that data protection, transparency and IP (copyright) are complied with, protecting your asset is one of our key benefits.

Protecting your question bank from security risks

Thirdly, lets consider security

Many users are jumping on the use of AI without the full knowledge of the consequences. ChatGPT and other LLMs are being used by staff, probably unknown to the organisation to accelerate test item development. Getting new questions banks ready can give a competitive advantage and time is always the defining factor. Any questions created by using a public LLM are now in the data memory on the Internet and your question bank is now compromised, That is why AI question generators like IQ have been provided by Assessment vendors, we understand the risks and its our way of helping secure your item banks and supporting the authoring teams by giving them the speed, accuracy and improved quality they are looking for with out the risks or backlash of exposed items. This ensures that nobody can search and find the question items online.

Minimising hallucinations and misinterpretations

Fourthly, What about hallucinations and misinterpretations

Many people have voiced their concerns about how AI gives out wrong results, strange ideas and totally inappropriate data. They worry about the ethical use of AI is sensitive information such as exam questions. This is why we advocate the use of iQ within your standard quality process. iQ is trained on the information you give it, ensuring that the chance of any spurious results is minimised.

How iQ works within your assessment workflow

iQ is an intelligent tool designed to enhance the productivity of item writers by streamlining the question creation process. Fine-tuned using your own content and data, iQ ensures that the questions it generates are relevant, contextualised, and aligned with your specific assessment needs.

Integrated directly into eNetAssess, item writers access iQ via a simple in-platform widget. From there, they can request a single question or a complete set, using prompts such as: “Write a question for Learning Outcome X1.”

In response, iQ quickly generates a tailored question and displays it on screen. Item writers can review the suggestion, make any necessary edits, or choose to use it as is. When satisfied, they can either export the question to a format like PDF for external review, or seamlessly insert it into the eNetAssess editing screen by selecting "Use Suggestion". From there, it's ready to be saved or supplemented with additional information.

Using iQ doesn't demand technical knowledge or AI experience. We handle all aspects of the large language model (LLM) training for you. The process is straightforward: simply provide us with a training set, this could be existing documents, question banks, or other relevant materials. We use this content to fine-tune a model that's uniquely tailored to your organisation.

Once your customised model is ready, we’ll walk you through a handover session with our technical team. After that, you're ready to begin generating high-quality, contextualised questions with ease, no “training wheels” needed.

iQ is a tool to solve a “real-world” problem.

Want to find out more?

If you’d like to find out more about how iQ can support your assessment authoring process, get in touch for a personalised demo - we’ll show you how it works and answer any questions you may have.