Skip to main content

LLM Management and Fine-tuning

There are 2 main components for managing models in Compressa.

Management API​

URL: http://your_address:5100/

Management API is a REST API for managing models.

With it you can:

  • Load a new model into Compressa
  • Start a model for inference
  • Fine-tune a model
  • Etc.

Detailed description of available commands is available on the Management API page.

Fine-tuning​

You can fine-tune models using LoRA/QLoRA both via UI interface and via REST API. In addition, a UI dashboard is available for monitoring the fine-tuning process and metrics

More about the model fine-tuning process on the special page.