Benchmarking performance of LLMs

AES uses Claude on Google Cloud’s Vertex AI to automate their safety audit process, employing a multi-agent system to analyze documents, break down tasks, and generate reports. This system processes hundreds of pages, evaluates compliance, and produces detailed audit reports in about an hour.

Local Falcon, a platform that helps businesses improve their Google search rankings, utilizes Anthropic's Claude to analyze millions of customer reviews. This enables them to deliver clear, actionable recommendations, enhancing their clients' local search visibility.

Grab, a leading food delivery and rideshare company in Southeast Asia, uses OpenAI’s GPT-4 with vision fine-tuning to process millions of street-level images collected from drivers, enabling it to accurately identify traffic signs and lane dividers and reduce manual mapping labor. This approach yields more reliable, hyperlocal map data that supports their ride-hailing, delivery services, and enterprise clients.

Roboflow uses Meta's Segment Anything Model (SAM) to enable users to automatically segment objects in images and videos, significantly reducing the time required to create training datasets for computer vision models.

Untukmu.AI, an online gifting site in Indonesia, uses Meta's Llama 3.1 8B model with split inference processing to protect customer privacy. By running part of the AI model on customers' devices and the rest on their servers, they deliver personalized gift recommendations without accessing or storing personal data. This ensures customer privacy while still providing high-quality, tailored suggestions, enhancing trust and satisfaction.

CodeGPT, a popular coding assistant with over 1.4 million downloads, integrates Meta's Llama models to enhance developer productivity. By using Llama 3.2 (90B), CodeGPT helps developers not just generate code but also answer questions about their codebase, debug code, and onboard new team members. It includes a codebase graph mechanism that lets Llama understand entire repositories, allowing developers to effectively "talk" with their code. This integration leads to at least a 30% increase in productivity and accelerates onboarding from months to days.

StudyFetch built a comprehensive learning platform powered by Claude, providing personalized learning experiences through their AI tutor Spark.E. The platform analyzes lectures, generates study materials, and offers 24/7 tutoring support in over 20 languages, enabling students to master complex subjects with tailored, interactive assistance.

Intercom, a leading customer service platform, integrated Claude from Anthropic into their AI agent Fin to automatically resolve up to 86% of customer support queries with human-quality, personalized responses in over 45 languages. This implementation reduces response times from 30 minutes to seconds and enables human agents to focus on more complex issues.

AES uses Claude on Google Cloud’s Vertex AI to automate their safety audit process, employing a multi-agent system to analyze documents, break down tasks, and generate reports. This system processes hundreds of pages, evaluates compliance, and produces detailed audit reports in about an hour.