[Docs] :Guide to Self-Hosting Refact.AI

[svsairevanth12]

This document provides a comprehensive, step-by-step process for setting up a self-hosted instance of Refact.AI using the official Docker method, with additional configuration details and troubleshooting tips.

Prerequisites

Before you begin, ensure you have the following installed on your host machine:

• Docker Engine: You can find the installation guide at [docs.docker.com/engine/install/](https://docs.docker.com/engine/install/).
• NVIDIA GPU Support for Docker:
  • For Linux: Install the NVIDIA Container Toolkit.
  • For Windows: Install WSL 2 (Windows Subsystem for Linux) and ensure Docker Desktop is configured to use the WSL 2 backend.

Step 1: Run the Docker Container

This single command will download the necessary Docker image, configure a persistent volume for your data, and launch the Refact server.

1. Open your command-line terminal (or PowerShell on Windows).
2. Execute the following command:

bash

docker run -d --rm -p 8008:8008 -v refact-perm-storage:/perm_storage --gpus all --shm-size=256m smallcloud/refact_self_hosting:latest

Command Breakdown:

• docker run: Starts a new Docker container.
• d: Runs the container in detached mode (in the background).
• -rm: Automatically removes the container when it is stopped.
• p 8008:8008: Maps port 8008 on your host machine to port 8008 inside the container.
• v refact-perm-storage:/perm_storage: Creates and mounts a persistent Docker volume named refact-perm-storage. This volume stores your models, logs, and configurations, ensuring data persistence.
• -gpus all: Grants the container access to all available NVIDIA GPUs.
• -shm-size=256m: Allocates 256MB of shared memory to the container, which is important for model loading and inference performance.
• smallcloud/refact_self_hosting:latest: The official Docker image for the Refact self-hosted server.

Note: The --shm-size=256m parameter is crucial for proper operation, as mentioned in the community discussions. This shared memory allocation helps prevent issues with model loading and improves performance.

Step 2: Access the Web Interface

With the container running, you can now access the web administration panel to manage your instance.

1. Open your web browser.
2. Navigate to: http://127.0.0.1:8008 (use your server's IP address if running on a remote machine).
3. The first user to register on this page will be designated as the administrator.
4. Use the Models tab to download your desired AI models and the Inference tab to activate them and assign them to specific tasks.

Model Configuration

Important Notes about Model Downloads:

• Use the Models tab to download your desired AI models from the web interface
• For gated models (models that require special permissions), you may need additional authentication or approval steps
• The system supports both open-source models and integration with third-party APIs like OpenAI and Anthropic
• On a self-hosted mode, you can also configure OpenAI and Anthropic API integrations. Go to Model Hosting page → 3rd Party APIs section and toggle the switch buttons for OpenAI and/or Anthropic

System Requirements

• GPU Memory: Ensure you have sufficient GPU memory for the models you plan to use
• CPU Alternative: Another option is to use it with CPU, but it'll be slower
• Check system requirements before choosing your model configuration

Step 3: Connect Your IDE

The final step is to configure your code editor's Refact plugin to communicate with your local server.

For VS Code:

1. Navigate to Extensions and search for Refact.ai Assistant.
2. Install the extension if not already installed.
3. Open the Extension Settings.
4. In the Infurl field, enter the address of your server: http://127.0.0.1:8008

For JetBrains IDEs (IntelliJ, PyCharm, etc.):

1. Navigate to Settings → Tools → Refact.ai.
2. Select the Advanced tab.
3. In the Inference URL field, enter the address of your server: http://127.0.0.1:8008

Additional Configuration Options

Privacy and Security Features

Refact allows you to restrict access to particular files or projects, ensuring that your private code or confidential files are protected. We never store your code on the server side

User Authentication

• The system supports user registration and login
• Install plugin for your IDE: JetBrains or VSCode and sign up or login in to your account
• You can also add user flows via login UI forms for better access control

Troubleshooting

Common Issues:

1. Container fails to start: Ensure Docker has GPU access and the NVIDIA Container Toolkit is properly installed
2. Models not loading: Check that you have sufficient GPU memory and the -shm-size parameter is set correctly
3. IDE connection issues: Verify the server URL is correct and the container is running on the expected port

Performance Optimization:

• Ensure adequate GPU memory allocation
• Consider adjusting the -shm-size parameter if you encounter memory-related issues
• Monitor system resources during model downloads and inference

Setup Complete

Your Integrated Development Environment (IDE) is now connected to your private, self-hosted Refact.AI instance. You have full control over your models and data, with an intuitive Web UI for management.

Additional Resources

• Documentation: Visit the official Refact.AI documentation for detailed configuration options
• Community Support: To get assistance from the Refact community and team members, join the Discord server
• GitHub Repository: Check the official GitHub repository for updates and community contributions
• Model Support: Refact offers a wholesome developer experience by making multiple functions available inside one IDE. At the core of Refact is our autocomplete feature that works with 20+ programming languages

This self-hosted setup ensures maximum privacy and control over your AI coding assistant, making it ideal for enterprise environments or developers who require complete data sovereignty.