Google Gemini support for LLM Playground and Evaluations via Vertex AI platform
Hassieb PakzadLangfuse now supports Google Gemini models via the Vertex AI for LLM Playground and Evaluations.
We’re excited that Langfuse now supports access to the Gemini model family via Google Vertex AI for both the LLM Playground and Evaluations. This integration allows users to leverage Google Vertex AI platform’s powerful language models directly within the Langfuse ecosystem. Whether you’re prototyping in the Playground or conducting LLM-as-a-judge evaluations, access to Gemini and other models on the Google Vertex AI platform enhances your toolkit for building and optimizing AI-powered features.
Key Features
-
LLM Playground Integration: You can now use the Gemini model familiy via Google Vertex AI models in the Langfuse LLM Playground. This enables quick experimentation and testing with various Vertex AI platform models.
-
Evaluations Support: Google Vertex AI models can be utilized in Langfuse Evaluations, allowing for comprehensive assessment and comparison of model performance.
-
Easy Setup: Adding your GCP service account JSON key is straightforward in the project settings, making it simple to get started with these new capabilities.
How to Get Started
To begin accessing Google Vertex AI models in Langfuse (assuming you have already activated the Vertex API in your GCP project settings):
- Create a new service account and assign the
Vertex AI Userrole to it in the GCP IAM settings - Create a JSON key for the newly created service account
- Navigate to the LLM API Keys section in your Langfuse settings.
- Add a new LLM API Key for the adapter
vertex-ai. Paste your service account’s JSON key into the secret key field. API keys are stored encrypted on our servers. - You’re all set! You can now select Google Vertex AI models in the LLM Playground and Evaluations.