Announcements

Stay informed about Portkey's newest releases, features, and improvements

  1. GPT-4o Mini

    New Feature

    GPT-4o mini is a game-changer for AI accessibility. It is now available on Portkey!

    ✅ 60% cheaper than GPT-3.5 Turbo

    ✅ Outperforms competitors on key benchmarks

    ✅ Multimodal: supports both text and vision

    image

    Like

  2. New Tool Choice Param: "Required"

    New Feature
    Prompts

    When using function calling, you can now ensure that the model 100% calls one of your specified functions by setting tool_choice: "required" from the right sidebar in prompts playground.

    CleanShot 2024-07-11 at 15.39.13@2x

    This param is currently supported for OpenAI models. Here's a detailed description for what each value of tool_choice does:

    CleanShot 2024-07-11 at 15.48.19@2x

     Check out the detailed guide by OpenAI here.

  3. Service Account-based Auth (JWT) for Vertex

    New Feature
    CleanShot 2024-07-09 at 09.31.41@2x

     

    Passing a new OAuth 2.0 access token every time you have to make a Vertex request is tiring right? Portkey now has a flawless solution for you - just add your Service Account JSON to Portkey vault once, and seamlessly route to Vertex AI models using Portkey's virtual key system!

    Head to the Virtual key tab and just add your Vertex details - get a corresponding key that you can use anywhere on Portkey.

    3

  4. Redesigned Logs Drawer

    Improvement
    CleanShot 2024-07-09 at 09.29.27@2x

    Cleaner, nicer, more easier to navigate! Do you like the redesign? React to this post and let us know!

     

    2

  5. Portkey is app is now a PWA

    New Feature

    You can install it easily as a standalone app, and keep it handy for all your LLM monitoring, prompts, and gateway needs!

    app-portkey

    6

  6. Important: Log & Prompt Restrictions on Dev Plan

    Announcement

    This week, we are starting to enforce the limits on Portkey's Dev plan as per our pricing, which includes:

    • 10,000 logs / month
    • 3 saved prompts
    • 3-day log retention

    For users who want more advanced features, upgrading to the Pro Plan is easy. We've made it more affordable - it starts at $49 now, from the earlier $99. Here's what's included in the Pro plan:

    • 100k logs / month
    • Unlimited prompts
    • Semantic caching
    • Role-based access control

    Just head to "Settings" -> "Billing" in the Portkey app and add your card details.

     

    Please Note: Your existing saved prompts (even if more than 3) will keep working, and your app will continue to function normally - NONE of your requests will ever fail. We'll stop logging requests beyond the 10k/month limit on the Dev plan though.

    4

  7. Simplified LLM Provisioning & Logging for Enterprise Teams

    New Feature

    New powerful features for enterprise-grade team management, provisioning, and logging.

    LLM Provisioning

    Specify provider, model, and other settings for each team in a Portkey Config and attach it as default to an API key. Admins can then share the Portkey API key with the respective team which will have the provisioned LLM, budget, rate limit, and more.

    Granular, Team-level Logging

    Attach default metadata with an API key that can contain key information about the team, service, and more. Any request made with this API key will also log the default metadata for better tracking, debugging, spend analysis, and more.

    Streamlined API Usage

    We now accept the Portkey API key in the "Authorization" header. Paired with default Config & metadata, we've completely eliminated the need to approve or manage Portkey-specific headers for your app.

    How it Works

    1. Admin Provisions: An admin generates a restricted Portkey API key, attaches a default Config with team-specific settings, and adds team-specific metadata for logging and tracking.
    2. Team Consumes: Developers within the team can now use the provisioned Portkey API key to make requests to the Portkey API, and avail all Portkey features.

     For more information, refer to the Portkey Documentation.

    Like

  8. Monitoring Agents for Production-Readiness with Portkey

    Community

    One of the key challenges of building AI agents is the lack of visibility into key performance metrics such as:

    • Number of API Requests: How many requests are being made to LLMs.
    • Token Usage & Costs: How many tokens are being consumed and the cost for each agent run.
    • Latency: How long is the agent taking to complete a set of tasks.

    We are sharing a cookbook that easily demonstrates how you can enhance your Autogen & CrewAI projects with full-stack telemetry and get the confidence to ship them to production.

     Autogen x Portkey | CrewAI x Portkey 

     

    Like