Introduction

The EdgeVPN.io configuration file uses the JSON format, and specifies the parameterized options for the various modules that make up the controller. A valid config.json file is required to run the software, with contents as described below. Each section describes one of the main configuration file’s modules, with snippets of JSON. At the end of this document, a complete configuration file example is shown.

CFx module

This module is used to configure the overall controller framework. It configures unique identifiers for the overlay and the node. Currently, a node can be bound to a single overlay ID

  • Model specifies a mnemonic to describe customized IPOP controllers. You can develop and add your own modules, or remove or replace existing modules. For the vast majority of uses, set it to “Default”

  • Overlays specifies the overlay ID, a 28-bit number encoded in hexadecimal format. Note: currently, EdgeVPN.io only supports a single overlay

  • NodeId specifies the node’s unique ID. NodeId is a 128-bit number also in hexadecimal format. You may specify a unique ID, or, if left blank, the framework will generate a random ID. The example below shows an overlay with ID “101000F” and NodeID “a100001feb6040628e5fb7e70b04f001”

  • NidFileName specifies a fully qualified file name for storing a framework-generated NodeId

  "CFx": {
    "Model": "Default",
    "Overlays": [
      "101000F"
    ],
    "NodeId": "a100001feb6040628e5fb7e70b04f001"
  },

Logger module

The controller logger module. Used by all other modules for logging. Supports disk file and console streams.

  • LogLevel specifies the desired level of logging. Must be one of (in order of verbosity): NONE, ERROR, WARNING, INFO, or DEBUG

  • ConsoleLevel specifies a separate logging level to be used for the console; applies when Device is set to All

  • Device specifies the output stream for logging. Supported values: File, Console, All

  • Directory specifies the directory where logs are to be stored

  • CtrlLogFileName specifies the name of the log file for the controller

  • TincanLogFileName specifies the name of the log file for the TinCan WebRTC tunnel datapath

  • MaxFileSize specifies the maximum size for individual log files

  • MaxArchives specifies the number of log files to archive before overwriting

  "Logger": {
    "LogLevel": "DEBUG",
    "Device": "File",
    "Directory": "/var/log/edgevpn/",
    "CtrlLogFileName": "ctrl.log",
    "TincanLogFileName": "tincan_log",
    "MaxFileSize": 10000000,
    "MaxArchives": 1
  },

TincanInterface module

This module configures parameters relevant to the connection between the controller and the TinCan module. These are not needed to be modified from defaults for the vast majority of use cases.

  • MaxReadSize specifies the maximum buffer size for Tincan Messages

  • SocketReadWaitTime specifies the socket read wait time for Tincan Messages

  • RcvServiceAddress specifies the controller server IPv4 address

  • SndServiceAddress specifies the Tincan server IPv4 address

  • RcvServiceAddress6 specifies the controller server IPv6 address

  • SndServiceAddress6 specifies the Tincan server IPv6 address

  • CtrlRecvPort specifies the controller listening port

  • CtrlSendPort specifies the Tincan listening port

Signal module

This model specifies how to connect to XMPP services to establish a signaling channel for bootstrapping the creation of tunnels

  • Enabled should be set to true

  • CacheExpiry specifies the minimum duration (in seconds) that an entry remains in the NodeID -> JID mapping cache

  • Overlays specifies each overlay to be configured. This needs to match overlays described in the CFx module (in the example below, “101000F”)

  • HostAddress specifies the IP address or host name of the XMPP server

  • Port specifies the port to connect to the MXPP server. This is usually 5222 for password-based authentication, and 5223 for certificate-based authentication

  • AuthenticationMethod specifies the method for authenticating the user with the XMPP server. Possible values: PASSWORD, x509

  • Username specifies the name of the XMPP user

For password-based authentication: AuthenticationMethod is PASSWORD

  • Password specifies the user’s password

Example:

  "Signal": {
    "Enabled": true,
    "Overlays": {
      "101000F": {
        "HostAddress": "A.B.C.D",
        "Port": "5222",
        "Username": "user@xmppsite.com",
        "Password": "password",
        "AuthenticationMethod": "PASSWORD"
      }
    }
  },

For certificate-based authentication: AuthenticationMethod is x509

  • CertDirectory specifies the directory where certificate and key are stored

  • CertFile specifies the name of the file storing the user’s certificate

  • KeyFile specifies the name of the file storing the user’s private key

Note: the port typically used for x509-based auth is 5223, not 5222

Example:

  "Signal": {
    "Enabled": true,
    "Overlays": {
      "101000F": {
        "HostAddress": "A.B.C.D",
        "Port": "5223",
        "AuthenticationMethod": "x509",
        "CertDirectory": "/home/user/edgevpn/cacerts/",
        "CertFile": "edgevpn.crt",
        "KeyFile": "edgevpn.key"
      }
    }
  },

Topology module

Module that defines and enforces the overlay’s topology. Currently, EdgeVPN.io supports a Symphony-based structured peer-to-peer topology for its tunnels

  • Overlays specifies a configuration for each overlay being managed by this controller. Each overlay is specified by its UUID - a hexadecimal value matching one in the CFx Overlays list (see above). Note: currently, the system supports only a single overlay.

  • Name a mnemonic string to name the overlay

  • Description a description string of the overlay

  • EnforcedEdges specifies a list of NodeIds for which that a tunnel should alwayes be created to. Optional for most deployments, as the topology manages links according to its own policies, but can be used if you would like to manually create links. Example: [ “1234..5”, 1234..6”, 1234..7” ]

  • ManualTopology specifies if the topology module should only create EnforcedEdges. Values: true or false (default)

  • PeerDiscoveryCoalesce specifies the number of new peer notifications to wait on before attempting to update the overlay edges. If this threshold is not reached, the overlay will be refreshed on its periodic TimerInterval

  • MaxSuccessors specifies the maximum number of successors links to create in the Symphony topology. These are the outgoing links that connect to the neighbors “to the right” in the overlay’s ring. Typically, values in the 2-4 range are sufficient. Larger values improve fault tolerance, but also generate more tunnel maintenance overhead

  • MaxLongDistEdges specifies the maximum number of outgoing long distance Symphony edges to initiate

  • MaxConcurrentEdgeSetup specifies the maximum number of edges to be created concurrently. This can help prevent a “stampede” effect when joining an existing large overlay

  • Role specifies whether this one acts as a full virtual switch node in an overlay (Switch), or as a Leaf node that joins the overlay by connecting to a Switch node. In Switch role, a node will perform switching operations and accept incoming connection requests to create edges. In Leaf role, a node is a client/edge device that must connect to a Switch node. Leaf nodes reject any incoming requests for an edge.

Example:

  "Topology": {
    "PeerDiscoveryCoalesce": 4,
    "Overlays": {
      "101000F": {
        "Name": "SymphonyRing",
        "Description": "Scalable Symphony Ring Overlay for Bounded Flooding.",
        "MaxSuccessors": 2,
        "MaxConcurrentEdgeSetup": 5,
        "Role": "Switch"
      }
    }
  },

LinkManager module

This module creates and manages the WebRTC based tunnels, which are the overlay edges between peers

  • Stun specifies a list of one or more STUN servers for NAT traversal. A deployment needs at least one STUN server configured in order to support NAT traversal. STUN server endpoints are specifies in the format address:port

  • Turn specifies a list of dictionaries, which specify the TURN server(s) and corresponding credentials. TURN servers are needed to allow nodes behind symmetric NATs to communicate, when STUN-based NAT traversal fails.

Address specifies the TURN server endpoint IP address and port, in format address:port

User specifies the TURN user name

Password specifies a corresponding TURN password for User

  • Overlays specifies a configuration for each overlay being managed by this controller, starting with the UUID (a hexadecimal value matching one in the CFx Overlays list (see above) Note: currently, the system only supports a single overlay.

  • Type currently, the only value allowed is TUNNEL

  • TapName specifies the prefix used for creating the tap virtual network interface devices. Each tunnel created by the link manager is bound to a tap device of the operating system; the full name of the tap device consists of this prefix, appended with the first 7 characters of the link ID (e.g. tnl-1234567)

  • IgnoredNetInterfaces specifies a list of TAP device names that should not be used for tunneling. No tunnel endpoints points to these network interfaces will be generated

  "LinkManager": {
    "Dependencies": [
      "Logger",
      "TincanInterface",
      "Signal"
    ],
    "Stun": [
      "stun.l.google.com:19302",
      "stun1.l.google.com:19302"
    ],
    "Turn": [{
      "Address": "w2.xirsys.com:80",
      "User": "your-user-id-string",
      "Password": "your-password-string"
     }],
    "Overlays": {
      "101000F": {
        "Type": "TUNNEL",
        "TapName": "tnl-"
      }
    }
  },

OverlayVisualizer module

This module configures information for an (optional) overlay visualizer service

  • Enabled specifies whether the visualizer is enabled (true) or not (false)

  • TimerInterval specifies the interval, in seconds, to send updates

  • WebServiceAddress specifies the IP:port endpoint of the visualizer’s Web service

  • GeoCoordinate specifies the geographical coordinates (lat,lon) of this node for display in the user interface

  • NodeName specifies the name of this node for display in the user interface

  "OverlayVisualizer": {
    "Enabled": false,
    "TimerInterval": 25,
    "WebServiceAddress": "192.168.0.42:5000",
    "GeoCoordinate": "14.073791,100.606308",
    "NodeName": "nd-001"
  },

BridgeController module

This module manages the network bridge interaction with the tap devices

  • Dependencies lists controller modules the BridgeController depends on. Currently, it depends on “Logger” and “LinkManager”

BoundedFlood

Under BoundedFlood, you configure parameters related to the implementation of broadcast/multicast over the P2P topology.

  • OverlayID is the 7-digit hexadecimal identifier for your overlay (e.g. 101000F)

  • LogDir is the directory where bounded flood logs go to (typically the same as the other logs, e.g. /var/log/edge-vpn/)

  • LogFilename is the file name for the bounded flood log (e.g. bf.log)

  • LogLevel is the logging level for this module (e.g. INFO, DEBUG)

  • MaxBytes maximum size of a log file archive

  • BackupCount number of archives for the log file

  • BridgeName specifies a mnemonic used for naming the bridge instance.

  • DemandThreshold specifies the amount of data transferred over a link that triggers the creation of an on-demand link (e.g. 100M for 100MBytes). Units supported included: K (Kilo), M (Mega) and G (Giga)

  • MonitorInterval Interval in which on-demand thresholds are monitored for

  • MaxOnDemandEdges maximum number of on-demand edges that this node may request; setting this to 0 disables creation of on-demand links

  • FlowIdleTimeout if an SDN flow rule is not being used (no packet maches the rule) by the SDN switch within this interval, it is removed

  • FlowHardTimeout a hard timeout for removing SDN flow rules (regardless of whether they are used or not)

  • MulticastBroadcastInterval interval in which a multicast sender will send broadcasts to program multicast trees

  • ProxyListenAddress listening address of the bridge module in the controller - the SDN controller uses this to send requests to the EdgeVPN.io overlay controller

  • ProxyListenPort listening TCP port of the bridge module in the controller - the SDN controller uses this to send requests to the EdgeVPN overlay controller

Overlays

  • Overlays specifies a configuration for each overlay being managed by this controller, starting with the UUID (a hexadecimal value matching one in the CFx Overlays list (see above). It holds configurations for the network interface device (NetDevice) and the SDN controller (SDNController) Note: currently, the system only supports a single overlay.

NetDevice

  • AutoDelete specifies whether to remove the bridge device that was specified when the controller shuts down. Possible values: True, False. Setting this to False is useful if you want the software to attach to an existing bridge device.

  • Type specifies the type of network bridge to instantiate. Supported values are OVS (for Open vSwitch), VNIC (virtual NIC), and LXBR (Linux bridge). For most use cases, OVS is used - a node connected in Switch role structured overlay requires OVS. You should only use VNIC if Topology’s Role is set to Leaf. You should onlye use LXBR when you have an overlay that is “hardwired” with ManualTopology (see Topology module section)

  • SwitchProtocol Use BF for Bounded Flood (default for Type OVS), STP for Spanning Tree Protocol (default for Type LXBR)

  • NamePrefix is the prefix used to name the primary bridge/switch; the prefix is appended with the Overlay ID string

  • IP4 specifies the IPv4 address to assign to the bridge. This is an optional parameter; if your deployment does not require assigning an IP configuration to the bridge, it can be omitted

  • PrefixLen specifies the network prefix length to apply to the bridge. (Optional parameter)

  • MTU specifies the maximum transmission unit size to be applied to the bridge

AppBridge

This optional section configures a secondary “patch through” bridge that is connected to the primary bridge in a configuration where the node runs an endpoint with its own IP address

  • AutoDelete specifies whether to remove the bridge device that was specified when the controller shuts down. Possible values: True, False (default)

  • NamePrefix is the prefix used to name this bridge; the prefix is appended with the Overlay ID string

  • IP4 specifies the IPv4 address to assign to the bridge. This is an optional parameter; if your deployment does not require assigning an IP configuration to the bridge, it can be omitted

  • PrefixLen specifies the network prefix length to apply to the bridge. (Optional parameter)

  • MTU specifies the maximum transmission unit size to be applied to the bridge. Optional parameter; by default, it is set to 1410

SDNController

This section configures the SDN controller endpoint

  • ConnectionType configures the transport used; set this to tcp

  • HostName configures the host where the SDN controller runs; set this to 127.0.0.1 (localhost)

  • Port configures the port where the SDN controller listens to; set this to 6333

Example:

  "BridgeController": {
    "Dependencies": [
      "Logger",
      "LinkManager"
    ],
    "BoundedFlood": {
      "OverlayId": "101000F",
      "LogDir": "/var/log/edge-vpn/",
      "LogFilename": "bf.log",
      "LogLevel": "INFO",
      "BridgeName": "edgbr",
      "DemandThreshold": "100M",
      "FlowIdleTimeout": 60,
      "FlowHardTimeout": 60,
      "MulticastBroadcastInterval": 60,
      "MaxBytes": 10000000,
      "BackupCount": 0,
      "ProxyListenAddress": "",
      "ProxyListenPort": 5802,
      "MonitorInterval": 60,
      "MaxOnDemandEdges": 0
    },
    "Overlays": {
      "101000F": {
        "NetDevice": {
          "AutoDelete": true,
          "Type": "OVS",
          "SwitchProtocol": "BF",
          "NamePrefix": "edgbr",
          "MTU": 1410,
          "AppBridge": {
            "AutoDelete": true,
            "Type": "OVS",
            "NamePrefix": "brl",
            "IP4": "10.10.10.21",
            "PrefixLen": 16,
            "MTU": 1410
          }
        },
        "SDNController": {
          "ConnectionType": "tcp",
          "HostName": "127.0.0.1",
          "Port": "6633"
        }
      }
    }
  }

UsageReport module

This configures an opt-in usage report module of a node. The usage report is a periodic event that posts non-identifying usage information to a service, and helps the EdgeVPN.io team collect gather information about usage of the software. The anonymized information reported is as follows:

  • The sha1 hash of the overlay ID
  • The sha1 hashes of the node ID of the reporting node, and of the node IDs of its peers
  • A monitonically increasing report ID

With this information, the EdgeVPN.io team can assess how many different networks and nodes have been deployed, and estimate uptimes. The use of cryptographic hashes ensures that overlay/node ID information is anonymized; the service does not log any information that can be used to identify a node, such as IP addresses.

By default, the usage report is not enabled. To enable it, the configuration file needs to explicitly set Enabled to true in this module. The TimerInterval (in seconds) specifies the period in which reports are posted to the endpoint specified in WebService

  "UsageReport": {
    "Enabled": true,
    "TimerInterval": 3600,
    "WebService": "https://qdscz6pg37.execute-api.us-west-2.amazonaws.com/default/EvioUsageReport"
  }

Additional information

The configuration file tells the controller framework which modules to load and with what parameters. If a module is not specified in the configuration, it is not loaded. Certain keys occur in multiple modules with the same meaning and effect. Each module has an Enabled key that, when set to false, will cause the controller framework to skip loading it. TimerInterval specifies how often a module’s timer event fires in seconds. Dependencies specify which modules are used a module and must be loaded before hand. In your installed EdgeVPN.io package, you will find in controller/framework/fxlib.py a file that contains the full set of default values in the CONFIG dictionary. This dictionary is loaded first, and any values specified in config file will override them. Therefore, changes should always be made to the config.json file and not the fxlib.py file.