Hugging Face (TGI) Configuration

This section explains how to configure a Hugging Face Text Generation Inference (TGI) model in AskSenseOps.

Prerequisites

Before configuring the model, ensure that your Hugging Face TGI endpoint is available and that you have the API key required to access it.

Adding a Hugging Face (TGI) Model

1. Open AskSenseOps.
2. Navigate to the model configuration section.
3. Click Add Model.
4. Enter the required configuration details.
5. Test the configuration and save the model.

Basic Info

Field	Description	Required
Display Name	Enter a name to identify the model in AskSenseOps.	Yes
Provider	Select Hugging Face (TGI).	Yes

Credentials

Field	Description	Required
API Key	Enter the API key used to authenticate with the Hugging Face TGI endpoint.	Yes

Model Configuration

Field	Description	Required
Model Name / ID	Enter the model identifier used by the endpoint.	Yes
Endpoint URL	Enter the URL of the Hugging Face TGI endpoint.	Yes

Use Fetch models, when available, to retrieve the models accessible through the configured endpoint.

Custom Headers

Custom headers are optional and can be used when connecting through a gateway that requires additional headers.

1. Under Custom Headers (Optional), click + Add.
2. Enter the required header name and value.
3. Add additional headers if needed.

Examples shown in the interface include x-openai-token, x-fabrix-client, and x-llm-model-id.

If a custom header carries the authentication token, the API Key field can be left empty.

Testing and Saving the Model

After entering the configuration details:

1. Review the display name, provider, API key, model identifier, and endpoint URL.
2. Click Test to verify the configuration.
3. If the test succeeds, click Save Model to save the configuration.

If the test fails, verify the API key, model identifier, endpoint URL, and any custom headers required by your endpoint.