Adjusting model parameters in LibreChat

Tags ai LibreChat

Questions

  • What are model parameters in LibreChat?
  • How do I change model parameters before sending a message?
  • What do terms like temperature, top P, and tokens mean?
  • Can I save my preferred parameter settings for later use?

Environment

This applies to Bowdoin College faculty, staff, and students using LibreChat on a desktop or laptop through a web browser. Model parameter adjustments are available for most AI models in LibreChat, though available settings can vary by model type.

Resolution

LibreChat offers a secure interface for prompting a variety of generative AI models along with the ability to use your documents to augment AI models' capabilities. It can serve as a centralized hub for all your AI conversations, providing a familiar, user-friendly interface enriched with advanced features and customization capabilities.

Adjusting model parameters in LibreChat

In LibreChat, you can adjust model parameters to fine‑tune how the AI responds to your prompts. Parameters can be changed in the chat interface before sending a message by opening the Parameters panel in the right hand customization column. This process is optional, and most people can leverage the default options for a satisfactory experience. 

Note that the settings might vary slightly for each model type, but many models will have the below configuration options: 

  • Max context tokens - Defines maximum input size. (1000 tokens is about 750 words)
  • Max output tokens - Defines maximum output size. (1000 tokens is about 750 words)
  • Temperature - Controls the “creativity" or randomness of the content being generated.
  • Top P - Alternative to temperature, refines the number of possible responses based on context provided.
  • Reasoning effort - For some models, increasing reasoning effort can help increase quality of responses, by indicating that more computational effort is required. Increased reasoning effort might increase the amount of processing time, as well.

 

Once you have made changes to the Parameter settings you can save them as a Preset whcih can then be accessed at any time from the Presets icon next to the Model name in the top left of the chat window:

Parameters in LibreChat

 

If you have questions, please contact the Service Desk at (207) 725-3030 or servicedesk@bowdoin.edu

Print Article