top of page

Frequently Asked Questions

Amelia’s AI Achieves Superior Performance in Aviation-Specific Tasks

Amelia 2.0 was rigorously benchmarked against GPT-4o, demonstrating higher accuracy and relevance in aviation-related scenarios.

Cosine Similarity

Amelia’s upgraded AI (version 2.0) outperformed GPT-4o in aviation-related question answering, using cosine similarity as the evaluation metric. This framework measures how closely the generated answers align with ideal responses. Amelia achieved a 10% improvement in similarity scores, ensuring that pilots and instructors receive more precise, actionable insights. This improvement enhances both training quality and decision-making by delivering aviation-specialized knowledge.

Amelia accurately outperforms GPT-4o

Private Pilot Oral Exam Benchmark

Through rigorous testing using 193 FAA oral exam questions, Amelia demonstrated higher accuracy than GPT-4o in covering critical regulatory and explanatory points. The benchmarking framework involved comparing each generated response to ideal answers, similar to the standards used by a Designated Pilot Examiner (DPE). Amelia scored 7% higher on average and achieved 15% more perfect answers, confirming its ability to meet professional pilot training standards with high accuracy and relevance.

Amelia outperformed FAA oral exam questions compared to GPT-4o
bottom of page