AI Developer Portal

The Developer portal is designed to enable your organisational users to be able to quickly adopt and use the AI tools that you have made available to them.

While Bots, AI Functions and Scripts are good, developers will most likely want to work with best-in-class tooling such as llangchain, and in doing so use vendor APIs directly.

This creates a problem, in that this interaction is in no way mediated or logged and if you are allowing access to your ingewted vector stores for Resource Augmented Generation, then there is no way to ensure that data kis not leaking into open platforms.

The AI Developer POrtal changes this, it enables users to work with the tools they want, but mediated through Montag by providing a proxy for popular vendor APIs.

How does it work?

A developer is given access to Montag with the “AI Developer Role”, they will be asked to create a new Application. This application will be given a unique ID and a token (which must be approved by an admin).

In the process of creating the applicationn they can select which data sources they may wish to query, the privacy rating of these data sources (or combination thereof) will determine which LLM API Client they can use.

The application will then provide unique API endpoints to the developer that they can drop into their tooling instead of the default vendor APIs.

Supported endpoints:

  1. /v1/chat/completions - The OpenAI completions API
  2. /v1/embeddings - The OpenAI embeddings API
  3. /query- The Piinecone Query API.

Example Python OpenAI usage:


from openai import OpenAI

client = OpenAI(
    api_key="APIKEY",
    base_url="http://localhost:8080/api/proxy/oai/1/default/v1"
)

response = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
    {"role": "user", "content": "Where was it played?"}
  ]
)

print(response.choices[0].message.content)

response = client.embeddings.create(
    input="Your text string goes here",
    model="text-embedding-ada-002"
)

print(response.data[0].embedding[:4])

Example Python pinecone usage:


import pinecone
from pinecone.core.client.configuration import Configuration as OpenApiConfiguration

h2 = "http://localhost:8080/api/proxy/pinecone/1"
k2 = "APIKEY"

openapi_config = OpenApiConfiguration.get_default_copy()
openapi_config.host = h2
openapi_config.verify_ssl = False
pinecone.init(api_key=k2, environment="default", openapi_config=openapi_config)

vecs = [
        -0.03266071155667305,
        0.0011330371489748359,
        -0.006271908991038799,
        # Concatenated for brevety
      ]


q = pinecone.Index("default").query(
    vector=vecs, 
    top_k=10,
    namespace="tyk-doc-semantic-005")
    

print(q)