Now that you have an agent working, you need to write tests to ensure that the quality of your answers don’t degrade as you add additional context.

To add a test to your agent, you can add the following to your .agent.yml file.

tests:
  - type: consistency
    n: 10
    task_description: "how many nights did I get high quality sleep?"

You can add as many tests as you’d like, for as many prompts as you like. For example:

tests:
  - type: consistency
    n: 10
    task_description: "how many nights did I get high quality sleep?"
  - type: consistency
    n: 10
    task_description: "how many hours do I sleep on average?"
  - type: consistency
    n: 10
    task_description: "what day do I typically get the most sleep?"

You can then run these tests using the following command:

oxy test my-agent.agent.yml

This will generate a final accuracy score and surface any consistency errors that the LLM detects.

At this point, you have a working agent as well as the ability to modify and test this agent. Congratulations!