Peer-to-Peer Apps in Go: Dial and Listen by Name

nfnet is the part of the nfltr Go SDK that connects two of your programs to each other by name. One side listens for a service, the other dials it, and each gets a net.Conn: no open port, no address to know, no NAT to configure. It is the same code the nfltr cat and nfltr up commands use.


What it is

The package is github.com/nfltr/relay/client/nfnet, in the Apache-2.0 module github.com/nfltr/relay/client:

go get github.com/nfltr/relay/client

It has two halves:

A *Conn is a net.Conn, so it works with io.Copy, bufio, net/http transports, gRPC dialers and anything else that takes one. Read deadlines, write deadlines and concurrent reads and writes behave as a TCP connection's do. Three methods go beyond net.Conn:

method what it does
CloseWrite() ends this side's writes: the peer reads io.EOF after the last byte, its own bytes still arrive
Reset() ends the session as failed: the peer reads an error, never io.EOF
Info() the session's target, id, path (relayed or direct), whether it is end-to-end encrypted, and for an accepted session who called

Writes block on the session's flow control, so a slow reader slows its writer and nothing piles up in memory. A slow session never holds up the others on the same connection.

A minimal program

An echo service and its client in one program. Run echo serve on one machine; it prints the target to dial. Run echo <machine>.echo/echo on another machine of the same account:

package main

import (
	"context"
	"crypto/tls"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"os"
	"path/filepath"
	"time"

	"github.com/nfltr/relay/client/machinecert"
	"github.com/nfltr/relay/client/nfnet"
)

// machineCert reads the certificate `nfltr config login` saved in dir
// (machine.json's chain, machine.key) at each handshake, so a renewal by
// the CLI takes effect.
func machineCert(dir string) func() (*tls.Certificate, error) {
	return func() (*tls.Certificate, error) {
		var saved struct{ Chain string `json:"chain"` }
		b, err := os.ReadFile(filepath.Join(dir, "machine.json"))
		if err == nil {
			err = json.Unmarshal(b, &saved)
		}
		if err != nil {
			return nil, err
		}
		cert, err := machinecert.ParseCert([]byte(saved.Chain))
		if err != nil {
			return nil, err
		}
		key, err := machinecert.LoadKey(filepath.Join(dir, "machine.key"))
		if err != nil {
			return nil, err
		}
		c := cert.TLS(key)
		return &c, nil
	}
}

func main() {
	home, _ := os.UserHomeDir()
	cfg := nfnet.Config{
		Relay: "nfltr.xyz:443", AgentRelay: "grpc.nfltr.xyz:443",
		Agent: "echo", ClientCert: machineCert(filepath.Join(home, ".config", "nfltr")),
	}
	if len(os.Args) < 2 {
		return
	}
	if os.Args[1] == "serve" {
		ln, err := nfnet.Listen(context.Background(), cfg, "echo", nfnet.ListenOptions{})
		if err != nil {
			log.Fatal(err)
		}
		defer ln.Close()
		fmt.Println("dial me as", ln.Addr())
		for {
			c, err := ln.Accept()
			if err != nil {
				return
			}
			go func() {
				defer c.Close()
				_, _ = io.Copy(c, c) // until the client's half close
			}()
		}
	}
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()
	c, err := nfnet.Dial(ctx, cfg, os.Args[1])
	if err != nil {
		log.Print(err)
		os.Exit(int(nfnet.CodeOf(err))) // 3 not found, 5 unreachable, ...
	}
	defer c.Close()
	_, _ = io.WriteString(c, "hello\n")
	_ = c.CloseWrite() // the server's copy ends; its bytes still arrive
	_, _ = io.Copy(os.Stdout, c)
}

The package's own documentation carries this program as its echo example, compiled with every release, and a two-way chat as its chat example.

ctx bounds the dial only. While the service's agent is offline or reconnecting, Dial asks again until ctx ends, so a client started before its server still connects. The session itself outlives ctx.

Targets

Dial takes one of four forms. ParseTarget reads them offline, with no network or DNS lookup, and anything else is a usage error.

form example what it reaches
<agent>/<service> echo/echo the agent named echo of this machine's account
<machine>.<agent>/<service> laptop.echo/echo the agent echo on the machine named laptop
<service>@<group> db@backend one machine of organisation group backend serving db; the relay picks the machine, the client checks it really is in the group
nf1… nf1… the one service a capability address names, on the relay the address names

An nf1 address is a secret: whoever holds it may open its service. The SDK never puts it in an error, an event or Info().Target; those show a short prefix of its id and its relay instead.

Who may connect

Every session goes through a relay, which checks who opens it and who serves it. A program authenticates to the relay with one credential:

How far the session is protected depends on the mode:

mode how it is set what it adds
plain Config.Org nil the relay's checks only; bytes are encrypted between each program and the relay, not end to end
organisation Config.Org from orgid.Load on a machine enrolled with nfltr org join TLS end to end between the two machines' certificates; a listener names the groups it admits (ListenOptions.AllowGroups), a client checks the server's name or group; the relay can forge neither side
capability address ListenOptions.Address (from nf1.Mint) on the server, the nf1 text on the client TLS end to end, pinned by the address; the client needs no credential of its own and no account

In organisation mode every session is end to end; DialOptions.Plain and ListenOptions.Plain reach or serve a plain service on purpose. The mode is always the program's own choice: the relay's answer never turns an end-to-end session into a plain one. See Organisations for enrolment and groups.

Relayed and direct paths

A session starts relayed: its bytes go through the relay. Organisation sessions and capability-address sessions may also move to a direct path between the two machines when both sides ask for one:

DirectMode what it allows
DirectOff relayed only (the SDK's zero value)
DirectOn a direct path over any address the two machines find, a LAN path first

The client sets it per dial (DialOptions.Direct), the server per service (ListenOptions.Direct). The move happens under a running session without losing, repeating or reordering a byte; when no direct path can be made (a strict NAT, a firewall), the session simply stays relayed. Info().Path and the path event say which path a session is on, and why it is relayed when it is.

Events

Config.Events receives each session's events as data, the same ones nfltr cat pipe prints as JSON lines:

kind when
open the other side accepted the session
path the session moved to a direct path or back (with the peer's address, or the reason it is relayed)
close the session ended cleanly, with the bytes each way
error the session failed, with its *Error
agent a listener's agent connected, is reconnecting, or was replaced

The callback runs on the session's own goroutine and must not block.

Errors

Every failure is an *nfnet.Error whose Code is the exit status of nfltr cat pipe, so a program can exit with it or branch on it. CodeOf(err) works on any error and returns 0 for nil, io.EOF and a cancelled context.

code name meaning
1 usage a target or option that is not valid
2 auth no credential, or the relay refused it (or revoked it during the session)
3 not_found the agent is offline or unknown, or does not serve that service
4 refused the service, its backlog or the relay's policy refused the session
5 unreachable the relay, or the service behind the agent, could not be reached
6 reset the session was cut after it opened
7 e2ee_pin_mismatch the server is not the identity the target names
9 balance_exhausted the serving account's prepaid balance is used up
10 e2ee_client_refused the server refused this client's identity

Error.Message names which hop refused and what fixes it; Error.Reason carries the relay's reason name when it gave one; errors.Is and errors.As see the cause through Unwrap. A session that ends cleanly reads io.EOF; a cut is never reported as io.EOF.

Ending sessions and listeners