166 lines
4.5 KiB
Markdown
166 lines
4.5 KiB
Markdown
# 03 - Remote Node Setup
|
|
|
|
_Follow these instructions on **every remote computer/server** you want to monitor. These machines will run lightweight exporters that gather hardware metrics, waiting for the Central Server to pull the data._
|
|
|
|
---
|
|
|
|
## 1. INSTALLING NODE EXPORTER (For standard hardware data)
|
|
_Node Exporter tracks CPU, RAM, Disk Space, and Network speeds._
|
|
|
|
**1. Download Node Exporter**
|
|
```bash
|
|
wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
|
|
```
|
|
|
|
**2. Extract Node Exporter**
|
|
```bash
|
|
tar -xvf node_exporter-1.7.0.linux-amd64.tar.gz
|
|
```
|
|
|
|
**3. Create a restricted system user for security**
|
|
```bash
|
|
sudo useradd --no-create-home --shell /bin/false node_exporter
|
|
```
|
|
|
|
**4. Copy the main engine to the secure Linux bin folder**
|
|
```bash
|
|
sudo cp node_exporter-1.7.0.linux-amd64/node_exporter /usr/local/bin/
|
|
```
|
|
|
|
**5. Hand ownership of the executable to the new user**
|
|
```bash
|
|
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
|
|
```
|
|
|
|
**6. Create the background service file**
|
|
```bash
|
|
sudo nano /etc/systemd/system/node_exporter.service
|
|
```
|
|
|
|
_Paste this exact code into the file and save:_
|
|
```ini
|
|
[Unit]
|
|
Description=Node Exporter
|
|
Wants=network-online.target
|
|
After=network-online.target
|
|
|
|
[Service]
|
|
User=node_exporter
|
|
Group=node_exporter
|
|
Type=simple
|
|
ExecStart=/usr/local/bin/node_exporter
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
**7. Turn Node Exporter on forever**
|
|
```bash
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl start node_exporter
|
|
sudo systemctl enable node_exporter
|
|
```
|
|
|
|
_(The Node Exporter is now running on port `9100`)._
|
|
|
|
---
|
|
|
|
## 2. INSTALLING NVIDIA GPU EXPORTER (Only for servers with GPUs)
|
|
|
|
_Note: The server must already have Nvidia drivers installed so the `nvidia-smi` command works in the terminal. Node Exporter does NOT monitor GPUs._
|
|
|
|
**1. Download the Exporter**
|
|
```bash
|
|
wget https://github.com/utkuozdemir/nvidia_gpu_exporter/releases/download/v1.4.1/nvidia_gpu_exporter_1.4.1_linux_x86_64.tar.gz
|
|
```
|
|
> **[WARNING] Panic `[ms]` Error:** Always use version `v1.4.1` or newer. Older versions crash on modern drivers because they output a unit called `[ms]`, which breaks Prometheus's strict naming rules.
|
|
|
|
**2. Extract it**
|
|
```bash
|
|
tar -xvf nvidia_gpu_exporter_1.4.1_linux_x86_64.tar.gz
|
|
```
|
|
|
|
**3. Create a restricted system user**
|
|
```bash
|
|
sudo useradd --no-create-home --shell /bin/false nvidia_exporter
|
|
```
|
|
|
|
**4. Copy the engine to the secure bin folder**
|
|
```bash
|
|
sudo cp nvidia_gpu_exporter /usr/local/bin/
|
|
```
|
|
|
|
**5. Hand ownership to the new user**
|
|
```bash
|
|
sudo chown nvidia_exporter:nvidia_exporter /usr/local/bin/nvidia_gpu_exporter
|
|
```
|
|
|
|
**6. Create the background service file**
|
|
```bash
|
|
sudo nano /etc/systemd/system/nvidia_gpu_exporter.service
|
|
```
|
|
|
|
_Paste this exact code into the file and save:_
|
|
```ini
|
|
[Unit]
|
|
Description=Nvidia GPU Exporter
|
|
Wants=network-online.target
|
|
After=network-online.target
|
|
|
|
[Service]
|
|
User=nvidia_exporter
|
|
Group=nvidia_exporter
|
|
Type=simple
|
|
ExecStart=/usr/local/bin/nvidia_gpu_exporter
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
**7. Turn it on forever**
|
|
```bash
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl start nvidia_gpu_exporter
|
|
sudo systemctl enable nvidia_gpu_exporter
|
|
```
|
|
|
|
_(The GPU Exporter is now running on port `9835`)._
|
|
|
|
---
|
|
|
|
## 3. ADDING THE REMOTE TARGET TO PROMETHEUS
|
|
|
|
Once your remote servers have their exporters running, you must configure the **Central Server** to scrape them.
|
|
|
|
### If the remote server is on the SAME network:
|
|
Simply use its Local IP Address. Go to your **Central Server** and edit `prometheus.yml`:
|
|
|
|
```yaml
|
|
- job_name: "office_desktop_1"
|
|
static_configs:
|
|
- targets: ["192.168.1.150:9100"] # For Node Exporter
|
|
- job_name: "office_desktop_1_gpu"
|
|
static_configs:
|
|
- targets: ["192.168.1.150:9835"] # For GPU Exporter
|
|
```
|
|
|
|
### If the remote server is across the internet (Ngrok/Cloudflare Tunnels):
|
|
If the server is remote and behind a firewall, you must expose the ports securely using a tunnel like Ngrok or Cloudflare.
|
|
|
|
For example, if you run Ngrok on the remote server pointing to port 9100:
|
|
```bash
|
|
ngrok http 9100
|
|
```
|
|
It gives you a URL: `https://my-custom-tunnel.ngrok-free.dev`.
|
|
|
|
Go to your **Central Server** and add it to `prometheus.yml` using `scheme: https`:
|
|
|
|
```yaml
|
|
- job_name: "remote_gpu_server"
|
|
scheme: https
|
|
static_configs:
|
|
- targets: ["my-custom-tunnel.ngrok-free.dev"]
|
|
```
|
|
|
|
> **IMPORTANT:** After editing `prometheus.yml`, always restart the Prometheus service on the Central Server: `sudo systemctl restart prometheus`.
|