LLMs in the IDE: The NETWAYS Managed AI Models API for data protection-compliant AI development

AI assistance directly in the IDE

The integration of Large Language Models (LLMs) in Integrated Development Environments (IDEs) has revolutionized software development. Developers benefit from intelligent code completion, automatic documentation and contextual suggestions directly in the editor. However, many cloud-based AI services raise questions about data protection, especially when working on sensitive projects.

The NETWAYS Managed AI Models API offers an attractive alternative here: an OpenAI-compatible API interface that is GDPR-compliant and provides both powerful language models (currently gpt-oss-120b) and embeddings (BAAI/bge-m3). In this article, we show how you can integrate this API into popular IDE plugins such as RooCode and Cline and become even more productive through semantic search.

Why a self-managed LLM API?

Data protection and GDPR compliance

The key advantage of the NETWAYS Managed AI-Models API is its GDPR compliance. Data that you share with the NETWAYS Managed AI-Models API is processed in GDPR- and ISO27001-compliant data centers in Germany and is never used for further training. This is often a critical point for companies that work with sensitive data or have to meet strict compliance requirements.

OpenAI compatibility

The API follows the OpenAI standard, which simplifies integration: plugins that were originally developed for OpenAI can be used with minimal adjustments. They use familiar endpoints such as /v1/chat/completions or /v1/embeddings.

Powerful models

• gpt-oss-120b: An open source model with 120 billion parameters for code generation, refactoring and documentation
• BAAI/bge-m3: A multilingual embedding model for semantic code search and retrieval augmented generation (RAG)

Setting up the API in IDE plugins

The configuration follows a standardized scheme for most plugins. All you need is

• API Base URL: The base URL of the Managed AI Models API https://api.ai.nws.netways.de/v1
• API Key: Your authentication token
• Model Name: The model ID (e.g. gpt-oss-120b)

Configuration for RooCode and Cline

For RooCode:

1. Open the extension settings in VS Code
2. Navigate to RooCode Settings
3. Select “OpenAI Compatible” as the provider
4. Enter the API Base URL: https://api.ai.nws.netways.de/v1
5. Add your API key
6. Set the model to gpt-oss-120b
7. Optional: deactivate the option Allow anonymous error and usage reports under About RooCode

For Cline:

1. Click on the Cline icon in the VS Code sidebar
2. Select “Settings” > “API Configuration”
3. API provider: “OpenAI Compatible”
4. Enter base URL, API key and model name same as for RooCode and optionally deactivate usage reports

For other OpenAI-compatible plugins: The general steps to take remain the same, most plugins offer a field for “Custom API Endpoint” or “OpenAI Compatible Provider”, where you can store these three required parameters.

Semantic search with embeddings: RAG for your codebase

The true strength of the Managed AI Models API becomes apparent when using Retrieval Augmented Generation (RAG). Instead of just giving the LLM the current code context, embeddings allow you to semantically search your entire codebase. To do this, you can use the local open source vector database Qdrant.

When a query is made, the search string is also converted into a vector. Qdrant then finds the k nearest neighbors (k-Nearest Neighbors, Top-K) in the vector space of your indexed codebase and returns the most relevant code snippets to RooCode, for example, which in turn passes them on to the text model as context.

Integration with RooCode and Qdrant

RooCode supports the indexing of your codebase and saves the embeddings in Qdrant. The complete process looks like this:

1. Index codebase: RooCode analyzes your files, splits them into meaningful chunks and creates embeddings with the help of an embedding model.
2. Save vectors: The high-dimensional vectors are stored with metadata (file path, line number) in Qdrant
3. Vector search: A query is also converted into a vector
4. k-Nearest Neighbor Retrieval: Qdrant finds the top-k most similar code sections in the vector space
5. Retrieval Augmented Generation: The LLM receives the code snippets found as additional context and generates an informed response

Start Qdrant with Docker

# Start Qdrant container
docker run -d \
  --name qdrant \
  -p 6333:6333 \
  -p 6334:6334 \
  -v $(pwd)/qdrant_storage:/qdrant/storage \
  qdrant/qdrant

# Verify it's running
curl http://localhost:6333/health

# The Qdrant Web UI is running at `http://localhost:6333/dashboard`.

After starting Qdrant, the vector database must be configured in RooCode and indexing must be initiated:

1. When RooCode is open, click on the database icon at the bottom right of the chat
2. Activate “Enable Codebase Indexing”
3. Enter the API Base URL https://api.ai.nws.netways.de/v1
4. Add your API key
5. Set the model to BAAI/bge-m3
6. Set the model dimension to 1024
7. Enter the Qdrant URL http://localhost:6333
8. Start indexing

Conclusion: AI assistance without compromise

The NETWAYS Managed AI Models API enables powerful AI integration into your development environment, without privacy concerns. OpenAI compatibility ensures smooth integration with popular tools, while embedding support enables semantic code search and RAG workflows.

With the described setup in RooCode, Cline and other plugins as well as the integration of Qdrant for vector search, you have a complete, GDPR-compliant AI pipeline for your day-to-day development work.