Files

kawsik b4acb8d1df initial commit

2026-06-02 10:37:57 +05:30

4.5 KiB

Raw Blame History

03 - Remote Node Setup

Follow these instructions on every remote computer/server you want to monitor. These machines will run lightweight exporters that gather hardware metrics, waiting for the Central Server to pull the data.

1. INSTALLING NODE EXPORTER (For standard hardware data)

Node Exporter tracks CPU, RAM, Disk Space, and Network speeds.

1. Download Node Exporter

wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz

2. Extract Node Exporter

tar -xvf node_exporter-1.7.0.linux-amd64.tar.gz

3. Create a restricted system user for security

sudo useradd --no-create-home --shell /bin/false node_exporter

4. Copy the main engine to the secure Linux bin folder

sudo cp node_exporter-1.7.0.linux-amd64/node_exporter /usr/local/bin/

5. Hand ownership of the executable to the new user

sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

6. Create the background service file

sudo nano /etc/systemd/system/node_exporter.service

Paste this exact code into the file and save:

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

7. Turn Node Exporter on forever

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter

(The Node Exporter is now running on port 9100).

2. INSTALLING NVIDIA GPU EXPORTER (Only for servers with GPUs)

Note: The server must already have Nvidia drivers installed so the nvidia-smi command works in the terminal. Node Exporter does NOT monitor GPUs.

1. Download the Exporter

wget https://github.com/utkuozdemir/nvidia_gpu_exporter/releases/download/v1.4.1/nvidia_gpu_exporter_1.4.1_linux_x86_64.tar.gz

[WARNING] Panic [ms] Error: Always use version v1.4.1 or newer. Older versions crash on modern drivers because they output a unit called [ms], which breaks Prometheus's strict naming rules.

2. Extract it

tar -xvf nvidia_gpu_exporter_1.4.1_linux_x86_64.tar.gz

3. Create a restricted system user

sudo useradd --no-create-home --shell /bin/false nvidia_exporter

4. Copy the engine to the secure bin folder

sudo cp nvidia_gpu_exporter /usr/local/bin/

5. Hand ownership to the new user

sudo chown nvidia_exporter:nvidia_exporter /usr/local/bin/nvidia_gpu_exporter

6. Create the background service file

sudo nano /etc/systemd/system/nvidia_gpu_exporter.service

Paste this exact code into the file and save:

[Unit]
Description=Nvidia GPU Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=nvidia_exporter
Group=nvidia_exporter
Type=simple
ExecStart=/usr/local/bin/nvidia_gpu_exporter

[Install]
WantedBy=multi-user.target

7. Turn it on forever

sudo systemctl daemon-reload
sudo systemctl start nvidia_gpu_exporter
sudo systemctl enable nvidia_gpu_exporter

(The GPU Exporter is now running on port 9835).

3. ADDING THE REMOTE TARGET TO PROMETHEUS

Once your remote servers have their exporters running, you must configure the Central Server to scrape them.

If the remote server is on the SAME network:

Simply use its Local IP Address. Go to your Central Server and edit prometheus.yml:

  - job_name: "office_desktop_1"
    static_configs:
      - targets: ["192.168.1.150:9100"] # For Node Exporter
  - job_name: "office_desktop_1_gpu"
    static_configs:
      - targets: ["192.168.1.150:9835"] # For GPU Exporter

If the remote server is across the internet (Ngrok/Cloudflare Tunnels):

If the server is remote and behind a firewall, you must expose the ports securely using a tunnel like Ngrok or Cloudflare.

For example, if you run Ngrok on the remote server pointing to port 9100:

ngrok http 9100

It gives you a URL: https://my-custom-tunnel.ngrok-free.dev.

Go to your Central Server and add it to prometheus.yml using scheme: https:

  - job_name: "remote_gpu_server"
    scheme: https
    static_configs:
      - targets: ["my-custom-tunnel.ngrok-free.dev"]

IMPORTANT: After editing prometheus.yml, always restart the Prometheus service on the Central Server: sudo systemctl restart prometheus.

4.5 KiB Raw Blame History