Have you always thought of contributing to Portkey but did not find the right time? Hacktoberfest is your opportunity! We are giving away Airpods Pro & Portkey swag to to community contributors this month.

Read this blog for more info: https://portkey.ai/blog/portkey-hacktoberfest-24/ 

The latest open source Llama 3.2 models by Meta - 1B & 3B Instruct and 11B & 90B Vision are now live on Portkey!

Use them through Portkey's integrations with Fireworks AI, Groq, Together AI, and AWS Bedrock providers.

OpenAI's Assistants API v1 beta shuts down by Dec 18, 2024. It's time to migrate to v2 beta, which offers:

- 500x more file capacity

- Faster processing

- Enhanced search with reranking

- Customizable chunking

- Query rewriting

If you are using Assistant's API, make sure you migrate to the v2 beta API before the deadline.

Portkey now natively integrates with  DeepSeek .

DeepSeek AI offers advanced open-source models like DeepSeek and DeepSeek-V2, known for their exceptional reasoning, coding, and multilingual performance.

 Link to docs 

We're thrilled to share that now Portkey is an official partner of MongoDB to help you take your AI apps to production. With the combo of Portkey + MongoDB, you get:

→ Streamlined data management

→ Enhanced LLM API control

→ Faster prototype-to-production pipeline

→ Improved security and scalability

→ Cost-effective AI operations

All this over secure, scalable, and private deployments to propel your AI apps into production.

Read more on the partnership here: https://cloud.mongodb.com/ecosystem/portkey-ai 

Anthropic recently released a guide on Contextual Retrieval, a method that cuts RAG retrieval failures by up to 67%.

🔥We've adapted this powerful technique to work with Portkey's AI Gateway.

Now you can easily implement Contextual Retrieval across 250+ language models with just one interface.

Key benefits of Portkey's integration:

1. Access to multiple language models through a single API
2. Simplified key management process
3. Effortless switching between providers

 Link to the cookbook 

OpenAI's new o1-preview and o1-mini models are supported on Portkey.

Compared to gpt-4o, these models reflect for a long time, and are able to answer questions like "how many r's in the word strawberry?", "how many words in your output?" exceptionally well.

o1 models work significatly better on maths, science, puzzle solving, and coding tasks.

Try them on Portkey:

Portkey now natively integrates with Inference.net, which is a "wholesaler" of LLM inference tokens for open source models like Llama 3. Inference.net is 50-90% cheaper than leading inference providers in the market, and could be especially useful for batch inference jobs!

 Explore integration docs here.

This integration makes your DSPy pipelines production-ready, providing detailed insights into costs and performance metrics for every run. Plus, your existing DSPy code can now work across 250+ LLMs!

Learn more: DSPy docs   

We’ve also released a Production Guide based on Ganaraj Parmuda's talk at the LLMs in Prod BLR Meetup. Check it out here: DSPy Blog Guide 

Portkey Guardrails for JSON Keys & JSON Schema now have improved functionality:

✅ Detect JSON within code blocks and in plain text

✅ Improved JSON extraction to handle multiple JSON objects within a single response

✅ Enhanced reporting to return more informative data objects, including:

• Matched JSON
• Present/missing keys (for JSON Keys plugin)
• Validation errors (for JSON Schema plugin)
• Explanatory messages for successful and failed validations

Try it out now! If you don't have access to the Guardrails feature, ping us on Discord to enable it for your org.