Build a Secure, Always-On Local AI Agent with OpenClaw and NVIDIA NemoClaw

In recent years, AI agents have evolved from simple question-and-answer systems into sophisticated autonomous assistants capable of managing complex workflows. However, deploying these agents in a secure manner is crucial, especially when considering the risks associated with third-party cloud infrastructures. This article provides a comprehensive guide on building a secure, always-on local AI agent using NVIDIA’s NemoClaw and OpenClaw technologies.

Overview of NVIDIA NemoClaw

NVIDIA NemoClaw is an open-source stack designed for the secure, on-premises deployment of autonomous AI assistants. It utilizes NVIDIA’s Nemotron 3 Super models and is orchestrated by NVIDIA OpenShell and OpenClaw, which facilitate sandboxed execution and tool integration. This combination ensures that sensitive data remains on local devices, providing enhanced privacy and control.

Key Features of NemoClaw

• Sandboxed Execution: OpenShell provides a secure runtime environment that isolates the AI agent from external threats.
• Real-Time Policy Approval: External access requests are managed through real-time policy checks to ensure security.
• Local Inference: All data processing occurs locally, preventing sensitive information from being transmitted over the internet.
• Integration with Messaging Platforms: The system can connect to various messaging platforms, allowing for remote access and interaction.

Deployment Steps

This tutorial will guide you through the deployment of NemoClaw on an NVIDIA DGX Spark system. The steps include configuring the runtime environment, setting up Docker and Ollama, downloading the model, and integrating with Telegram for remote access.

Prerequisites

Before starting the setup, ensure the following requirements are met:

• Hardware: A DGX Spark (GB10) system running Ubuntu 24.04 LTS with the latest NVIDIA drivers.
• Docker: Version 28.x or higher, configured with the NVIDIA container runtime.
• Ollama: Installed as the local model-serving engine.
• Telegram Bot Token: Created through Telegram’s @BotFather.
• Estimated Time: Approximately 20–30 minutes for setup, plus an additional 15–30 minutes for the initial model download (around 87 GB).

System Verification

Run the following commands to verify system readiness:

• head -n 2 /etc/os-release (Expected output: Ubuntu 24.04)
• nvidia-smi (Expected output: NVIDIA GB10 GPU)
• docker info --format '{{.ServerVersion}}' (Expected output: 28.x+)

NemoClaw Components

Understanding the components of the NemoClaw stack is essential for building a secure AI assistant:

Configuring the Runtime Environment

To support GPU-accelerated containers, several Docker configuration steps are required:

1. Register the NVIDIA container runtime with Docker:

sudo nvidia-ctk runtime configure --runtime=docker

2. Set the cgroup namespace mode to host:

sudo python3 -c "import json, os; path = '/etc/docker/daemon.json'; d = json.load(open(path)) if os.path.exists(path) else {}; d['default-cgroupns-mode'] = 'host'; json.dump(d, open(path, 'w'), indent=2)"

3. Restart Docker:

sudo systemctl restart docker

4. Verify the NVIDIA runtime:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

5. Add the current user to the Docker group:

sudo usermod -aG docker $USER

Installing Ollama

Ollama is a lightweight model-serving engine for running large language models locally. Install it using the following command:

curl -fsSL https://ollama.com/install.sh | sh

Configure Ollama to listen on all interfaces:

sudo mkdir -p /etc/systemd/system/ollama.service.d
printf '[Service]nEnvironment="OLLAMA_HOST=0.0.0.0"n' | sudo tee /etc/systemd/system/ollama.service.d/override.conf
sudo systemctl daemon-reload
sudo systemctl restart ollama

Verify that Ollama is running and reachable:

curl http://0.0.0.0:11434