Evaluate AI Agent Response Relevance using OpenAI and Cosine Similarity

About

This n8n template demonstrates how to calculate the evaluation metric "Relevance" which in this scenario, measures the relevance of the agent's response to the user's question. The scoring approach is adapted from the open-source evaluations project RAGAS and you can see the source here https://github.com/explodinggradients/ragas/blob/main/ragas/src/ragas/metrics/_answer_relevance.py How it works This evaluation works best for Q&A agents. For our scoring, we analyse the agent's response and a