The ability to use large language models (LLMs) against your own data sets with Azure OpenAI is a game changer for enterprises looking to make use of these revolutionary AI tools while maintaining secure environments.
In this post, we are going to look at how an organisation can bring their own data into Azure OpenAI in a secure manner, giving them the ability to use a large language model (LLM) such at ChatGPT, but based on their own data. Before we get into it, you need to already have been approved for Azure OpenAI access and have an Azure OpenAI Service resource with either the gpt-35-turbo or the gpt-4 models deployed.
Azure OpenAI on your data supports the following types of files:
.txt.md.html- Microsoft Word files
- Microsoft PowerPoint files
Adding your own data source
To access your Azure OpenAI resource, go to Azure OpenAI Studio and sign in with your account credentials. Choose the proper directory, Azure subscription, and Azure OpenAI resource during or after the sign-in process. Then select the Chat playground tile after logging in.
Then on the Assistant setup tile, select Add your data (preview), followed by + Add a data source, as shown in the screenshot below.
In the pane that appears, select Upload files under Select data source. Select Upload files. Azure OpenAI needs both a storage resource and a search resource to access and index your data. For Azure OpenAI to access your storage account, you will need to turn on Cross-origin resource sharing (CORS). If CORS isn’t already turned on for the Azure Blob storage resource, select Turn on CORS.
Select your Azure Cognitive Search resource, or create a new one if required, then select the acknowledgment that connecting it will incur usage on your account. Then select Next.
On the Upload files pane, select Browse for a file and select the files you want to upload. Then select Upload files. Then select Next. On the Data management pane, you can choose whether to enable semantic search or vector search for your index. Finally, review the information you have entered, if you are happy to continue, then press Save and close.
Exploring data with the chat playground
Start using the conversation playground to explore Azure OpenAI’s possibilities without writing any code. You can enter a prompt to generate a completion in the text area. You may rapidly experiment and iterate with the capabilities from this page.
To enhance the effectiveness of your bot, you can experiment with design options like temperature and pre-response text. The REST API provides detailed information about each parameter.
Deploying your model
When you’re happy with how it looks in Azure OpenAI studio, click the Deploy to button to launch a web app right from the studio.
If you choose to deploy a web app, the first time you deploy a web app, you should select Create a new web app. Enter a name for the app, which will become part of the URL, this will be a normal Azure App Service style URL.
Summary
Of course, this is a simple example, but you can see how you can ingest data to train a generative AI model. You can use this for any type of supported data within your enterprise, and it can go towards any number of use cases. For additional security, you can configure access through your virtual network, denying traffic from the Internet. You can and should also use managed identities to add further security.
Martyn Coupland 