Benchmarking performance of LLMs

wealthAPI implemented a next‐gen contract detection solution by integrating DataStax Astra DB on Google Cloud and leveraging Google Gemini models for AI‐powered analysis. They deployed DataStax’s vector search and real‐time insights capabilities to scale contract detection across millions of users in less than three months, streamlining wealth management workflows by dramatically reducing response times and efficiently handling massive data volumes.

Aura Intelligence integrated Anthropic's Claude via Amazon Bedrock into its data pipeline to automatically classify over 200 million job titles and industry pairings from multi-language data, replacing manual lookups and fuzzy matching. They fine-tuned foundation models on proprietary datasets and leveraged AWS infrastructure, including SageMaker and prompt management, to automate QA, report generation, anomaly detection, and real-time hiring trend analysis.

LaunchNotes leverages Claude in Amazon Bedrock in their product 'Graph' to transform engineering data into actionable insights. Graph functions as an ETL platform with Claude managing data pipelines, helping engineering managers understand development metrics, reduce incident identification time, automate updates, and generate customized release notes and technical documentation.

Roboflow uses Meta's Segment Anything Model (SAM) to enable users to automatically segment objects in images and videos, significantly reducing the time required to create training datasets for computer vision models.

Untukmu.AI, an online gifting site in Indonesia, uses Meta's Llama 3.1 8B model with split inference processing to protect customer privacy. By running part of the AI model on customers' devices and the rest on their servers, they deliver personalized gift recommendations without accessing or storing personal data. This ensures customer privacy while still providing high-quality, tailored suggestions, enhancing trust and satisfaction.

CodeGPT, a popular coding assistant with over 1.4 million downloads, integrates Meta's Llama models to enhance developer productivity. By using Llama 3.2 (90B), CodeGPT helps developers not just generate code but also answer questions about their codebase, debug code, and onboard new team members. It includes a codebase graph mechanism that lets Llama understand entire repositories, allowing developers to effectively "talk" with their code. This integration leads to at least a 30% increase in productivity and accelerates onboarding from months to days.

Quillit integrated Anthropic’s Claude to automate qualitative research tasks by summarizing interview transcripts, generating contextual citations, and threading conversation data into comprehensive reports. They implemented the AI tool into their existing research workflow within three months, streamlining report writing, transcription, and analysis while ensuring data security and high precision.

NVIDIA partnered with Google Cloud to enable on-premises agentic AI by integrating Google Gemini models with NVIDIA Blackwell platforms and Confidential Computing, ensuring data sovereignty and regulatory compliance for sensitive enterprise operations. The solution further optimizes AI inference and observability by deploying a GKE Inference Gateway alongside NVIDIA Triton Inference Server, NVIDIA NeMo Guardrails, and NVIDIA Dynamo to enhance secure routing and load balancing for enterprise workloads.

Quantium deployed Anthropic's Claude across its organization to empower over 1200 employees in functions such as coding, proposal drafting, training development, and leadership coaching. They implemented the AI solution by launching an "ALL IN on AI" strategy with clear guidelines, practical guardrails, and comprehensive hands-on training programs integrated into daily workflows. This approach streamlined routine tasks and enabled teams to focus on strategic initiatives.