Install on Proxmox VE

ImageOVA
InventoryProxmox VE API
FlowsNetFlow from Open vSwitch, or sFlow from hsflowd, set on each node
Durationabout 45 minutes

Beta

The support of Proxmox VE is in beta.

Before you start

You need:

  • the DC Netscope OVA, downloaded from easyvirt.com,
  • Proxmox VE 8.2 or later for the import wizard (older versions can import the OVA from the command line),
  • a shell access (SSH or console) to each Proxmox VE node whose traffic you want to see,
  • an IP address for DC Netscope, or a DHCP network,
  • these network openings:
Flow From To Port
Web interface Your users DC Netscope 443/tcp
Inventory DC Netscope A Proxmox VE node 8006/tcp
NetFlow (Open vSwitch bridges) Proxmox VE nodes DC Netscope 9000/udp
sFlow (Linux bridges) Proxmox VE nodes DC Netscope 10000/udp

Which bridges do your nodes use?

How the flows are exported depends on the type of the bridges the virtual machines are connected to. In the Proxmox VE interface, open a node, then System > Network: the Type column reads Linux Bridge or OVS Bridge.

Bridge type Flows Software to install on the nodes
OVS Bridge NetFlow, exported by Open vSwitch None
Linux Bridge (the default of Proxmox VE) sFlow, exported by hsflowd hsflowd

The SDN zones of Proxmox VE (Simple, VLAN, VXLAN, EVPN) are built on Linux bridges: use hsflowd.

Import the OVA

With the import wizard (Proxmox VE 8.2 and later)

  1. Let a storage hold the files to import: open Datacenter > Storage, edit a file-based storage (local, for instance), and add Import to its Content.

  2. In the tree, select this storage under the node that will run DC Netscope, open its Import tab, click Upload and choose the OVA file.

  3. Select the uploaded OVA and click Import. In the wizard:

    • pick a VM ID, a name and the storage of the disks,
    • in the Advanced tab, set the network interface on the bridge of your choice, with the model VirtIO,
    • leave Start after import unchecked.

From the command line

Copy the OVA to the node, extract it, and import its OVF descriptor, with 150 as the identifier of the new virtual machine and local-lvm as its storage here:

mkdir /tmp/dcnetscope && tar -xf dcnetscope.ova -C /tmp/dcnetscope
qm importovf 150 /tmp/dcnetscope/*.ovf local-lvm
qm set 150 --net0 virtio,bridge=vmbr0

Remove /tmp/dcnetscope once the import is done.

To confirm

The controller and the network card models the appliance boots with on Proxmox VE (VirtIO SCSI and VirtIO network are expected to work), and whether the import wizard keeps the disk controller of the OVA.

Do not start it yet: its settings come first.

Set the network and the administrator account

DC Netscope reads its settings from cloud-init. Proxmox VE gives them to the virtual machine through a cloud-init drive and a snippet: a YAML file stored on a storage whose Content includes Snippets.

  1. On the node, write the settings in /var/lib/vz/snippets/dcnetscope.yml (the snippets folder of the local storage):

    #cloud-config
    email: admin@example.com
    os_password: a-strong-password
    ip_mode: static
    static_ip: 192.168.1.50
    static_netmask: 255.255.255.0
    static_gateway: 192.168.1.1
    dns_ip: 192.168.1.1
    • email and os_password create the first administrator of DC Netscope. The password is also the one of the root account of the virtual machine.
    • ip_mode is dhcp, static, or cloud-init to let cloud-init configure the network itself. The static_* fields are ignored in dhcp mode.
  2. Add a cloud-init drive to the virtual machine, and give it the snippet:

    qm set 150 --ide2 local-lvm:cloudinit
    qm set 150 --cicustom "user=local:snippets/dcnetscope.yml"

See Appliance settings for all the settings.

To confirm

That the appliance reads a snippet given with cicustom (NoCloud datasource). Without cloud-init, write the same settings in /etc/dcnetscope.yml from the console of the virtual machine, and reboot it: see the manual configuration.

Start the virtual machine. After a few minutes, its console shows the address of DC Netscope: open https://<DC Netscope address> in your browser and log in with the account you defined.

Create an account for DC Netscope

DC Netscope reads the inventory (nodes, virtual machines, containers, networks, tags and pools) with the API of Proxmox VE. An API token of a read-only user is recommended. On one node of the cluster:

  1. Create a role with the rights DC Netscope needs, and a user holding this role on the whole cluster:

    pveum role add DCNetscope --privs "Sys.Audit,VM.Audit,VM.Monitor,SDN.Audit,Pool.Audit,Datastore.Audit"
    pveum user add dcnetscope@pve
    pveum acl modify / --users dcnetscope@pve --roles DCNetscope
  2. Create an API token for this user. --privsep 0 gives the token the rights of the user:

    pveum user token add dcnetscope@pve netscope --privsep 0

    Write down the value it prints: it is the secret of the token, and is shown only once.

To confirm

VM.Monitor lets DC Netscope ask the QEMU guest agent for the IP addresses of the virtual machines on Proxmox VE 8. Proxmox VE 9 replaces it with finer rights (VM.GuestAgent.*): the list of rights depends on the version.

The IP addresses of the virtual machines

DC Netscope names a flow after the virtual machine owning its IP address. Proxmox VE knows the addresses of the containers, but the ones of a virtual machine only when the QEMU guest agent is installed in it and enabled in its Options (or when its address is static, set by cloud-init). The flows of the other virtual machines show IP addresses, which you can name in the External IPs view.

Connect Proxmox VE

  1. Open the administration interface, and go to the Services tab.

    Services tab

  2. In Connected infrastructures, choose Proxmox VE (beta), and type the address of a node (pve1.example.com, or pve1.example.com:8006). Then:

    • with an API token, type dcnetscope@pve!netscope as the user (the user, !, and the name of the token), and the secret of the token as the password,
    • with a user and a password, type the user with its realm (root@pam, dcnetscope@pve): without a realm, DC Netscope uses @pam.

    Click Add a new connector.

  3. Check that the status of Proxmox VE is green.

One connector covers a whole cluster: any of its nodes answers for the others.

Send the flows of the nodes

Proxmox VE has no central setting for the export of flows: it is set on each node, from its shell. Follow the tab of the type of your bridges.

OVS bridges: NetFlow from Open vSwitch

Open vSwitch exports NetFlow natively, as on XCP-ng.

  1. List the bridges of the node, and spot the ones carrying the traffic of your virtual machines (vmbr0 here):

    ovs-vsctl show

    The tap ports are the network interfaces of the virtual machines (tap101i0: the first interface of the virtual machine 101), the veth ports the ones of the containers.

  2. Send the flows of the bridge to DC Netscope, 192.168.1.50 here:

    ovs-vsctl -- --id=@nf create NetFlow \
        targets=\"192.168.1.50:9000\" \
        active_timeout=60 \
        add_id_to_interface=true \
        -- set Bridge vmbr0 netflow=@nf
    • targets: the address of DC Netscope and the port of its NetFlow collector,
    • active_timeout: how often, in seconds, the flows still open are exported,
    • add_id_to_interface: adds the identifier of the interface to the exported flows.
  3. Check the export:

    ovs-vsctl list netflow

    The targets line should read ["192.168.1.50:9000"].

Repeat for every bridge and every node.

Keep the export after a reboot

Open vSwitch forgets this setting when the node reboots, and when the network configuration is applied again from the Proxmox VE interface. Run the command from a systemd service started after the network, for instance /etc/systemd/system/dcnetscope-netflow.service:

[Unit]
Description=NetFlow export of Open vSwitch towards DC Netscope
After=network-online.target openvswitch-switch.service
Wants=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/ovs-vsctl -- --id=@nf create NetFlow targets=\"192.168.1.50:9000\" active_timeout=60 add_id_to_interface=true -- set Bridge vmbr0 netflow=@nf

[Install]
WantedBy=multi-user.target

and enable it with systemctl enable --now dcnetscope-netflow.service.

To confirm

Whether an ovs_extra line on the bridge in /etc/network/interfaces keeps the export across a network reload, which would avoid the service.

Linux bridges: sFlow from hsflowd

The Linux bridge exports no flows by itself. hsflowd (Host sFlow) is a small agent that samples the packets of the network interfaces of the node and sends them to DC Netscope in sFlow.

Activate the sFlow collector first

The sFlow collector of DC Netscope does not run by default: in the appliance settings, set Sflow Collector workers count to 1 or more. It listens on 10000/udp.

  1. Install hsflowd on the node. Download the Debian package matching the version of Proxmox VE (Debian 12 for Proxmox VE 8, Debian 13 for Proxmox VE 9) from the releases of host-sflow, then:

    apt install ./hsflowd-debian12_<version>_amd64.deb
  2. Write its settings in /etc/hsflowd.conf, with DC Netscope at 192.168.1.50 here:

    sflow {
      sampling = 256
      sampling.10G = 256
      polling = 30
      collector { ip = 192.168.1.50 udpport = 10000 }
      psample { group = 1 egress = off }
      dent { sw = on switchport = ^(tap|veth)[0-9]+i[0-9]+$ }
    }
    • sampling: one packet out of 256 is sampled. Lower it on a quiet node to see more of the small flows, raise it on a busy one to spare its CPU. sampling only applies to the interfaces of unknown speed: the tap interfaces report 10 Gb/s, and hsflowd gives them its own default of one packet out of 10000 unless sampling.10G is set,
    • collector: the address of DC Netscope and the port of its sFlow collector,
    • dent: samples the interfaces whose name matches switchport: the tap interfaces of the virtual machines and the veth interfaces of the containers. The interfaces of the virtual machines started later are sampled too. sw = on samples in software: with sw = off, hsflowd asks for a sampling in hardware, which the tap interfaces do not support, and no flow is exported.
  3. Start hsflowd, and have it started with the node:

    systemctl enable --now hsflowd

Repeat on every node.

Why the interfaces of the virtual machines, and not the bridge

Sampling vmbr0 itself shows mostly the traffic of the node: the traffic between two virtual machines of the same node goes from one tap interface to the other without crossing it. Sampling the packets each virtual machine sends (egress = off) sees every flow once, including the ones that never leave the node.

To confirm

This setup was validated with the virtual machines of a Proxmox VE 9.2 node and hsflowd 2.1.17, including a virtual machine with the firewall of Proxmox VE enabled: its tap interface is sampled, the fwbr, fwpr and fwln interfaces are left out by switchport. The veth interfaces of the containers remain to be validated.

Note

The firewall of DC Netscope, the firewall of the nodes, and the ones between the nodes and DC Netscope must let 9000/udp (NetFlow) or 10000/udp (sFlow) through.

Your physical switches can also send their flows, as a complement or as a start before installing anything on the nodes: see Network devices. They see only the traffic between the nodes, and with the VXLAN or EVPN zones of the SDN, this traffic is encapsulated.

Check that the flows arrive

Open the Dashboard. Within a few minutes, the collect summary shows the traffic received from the nodes, and the flows of your virtual machines appear with their names.

Summary of the collect

Nothing shows up? Check that:

  • 9000/udp or 10000/udp is open from the nodes to DC Netscope,
  • with Open vSwitch, ovs-vsctl list netflow shows the export on the nodes (a rebooted node may have lost it),
  • with hsflowd, systemctl status hsflowd shows it running, and Sflow Collector workers count is 1 or more,
  • with hsflowd, tc -s filter show dev tap<vmid>i0 ingress on a node shows a sample action with the expected rate (1/256, not 1/10000), and journalctl -u hsflowd shows no setRate(...) exitStatus=2. Without them, hsflowd sends only counters, and no flow appears,

and have a look at the troubleshooting page.

Flows appear with IP addresses instead of names? Their virtual machines probably have no QEMU guest agent: see the IP addresses of the virtual machines.

Next steps