replace the old rpc protocol by http rest?

[ThomasWaldmann]

borg 1.x uses rpc-over-ssh to access remote repositories. the protocol is so that the repository api is extended over the network.

borg2 has borgstore for most kinds of storage needs, including the rest backend / rest server combo that extends the borgstore backend api over the network. One usually would run that behind a reverse proxy that also adds TLS.

considering that, the old rpc code looks a bit superfluous/redundant now and we maybe could implement the ssh-based remoting with the http rest protocol instead of the rpc protocol. the server would then only expose the ssh port and ssh keys and forced commands could be used in the same way as with borg 1.x.

[ThomasWaldmann]

By Claude Opus 4.6 (thinking):

To support the transition from RPC-over-SSH to HTTP REST in Borg 2.0, here are the main ways to route HTTP traffic through an existing SSH connection:

1. Paramiko direct-tcpip Channel (Most programmatic)
   With Paramiko you can open a direct-tcpip channel on the existing SSH transport and use it as a socket for an HTTP client. This multiplexes HTTP over the existing SSH connection without needing extra ports:

channel = transport.open_channel('direct-tcpip', dest_addr=('127.0.0.1', 80), src_addr=('127.0.0.1', 0))
# Then wrap the channel in a socket-like object for the HTTP client

2. SSH Local Port Forwarding (-L)
   The standard ssh -L 8080:localhost:80 approach.

3. Unix Domain Socket Forwarding
   Forward a local socket to a remote TCP port or even a remote Unix socket:
   ssh -L /tmp/local.sock:/run/borgstore/rest.sock user@remote
   This is particularly useful if the REST server is socket-activated or restricted by file permissions.

4. SOCKS Proxy (-D)
   Running a SOCKS proxy over SSH and pointing the HTTP client to it.

5. ssh -W (Stdio Forwarding)
   Useful for piping traffic directly to a remote port, often used with socat or similar tools to create a local listener.

For Borg 2.0, the direct-tcpip channel approach seems most elegant as it avoids managing local ports and integrates directly into the Python networking stack.

[RonnyPfannschmidt]

Claude is completely feral again

Id recommend investigating http2 on stdio for ssh support

I recall multiple implementations support it

[ThomasWaldmann]

@RonnyPfannschmidt We would like to use requests on the client side and stdlib http server on the remote side, there is existing code for that in borgstore.

[ThomasWaldmann]

Maybe we can use UNIX Domain socket forwarding

openssh -L local-UUID.sock:remote-UUID.sock ...:

[requests_unixsocket]
         │
         ▼
[Local local-UUID.sock]  ◄─── Created & managed by local SSH client
         │
         ▼ (Encrypted SSH Tunnel)
         │
[Remote remote-UUID.sock] ◄─── Hit natively by remote SSH daemon
         │
         ▼
[Your Remote Python HTTP Server]

Notes:

• the local and remote sockets need to already exist when openssh -L is invoked and cleaned up afterwards
• path where to create the socket? ~ might be safe but needs to be expanded, not sure about /tmp or /run/UID
• more secure than tcp port fwd, also no port collisions
• needs requests-unixsocket or custom UNIXSocketAdapter

[RonnyPfannschmidt]

Just let Borgstore serve http2 on stdio and use a sane adaptor

[ThomasWaldmann]

@RonnyPfannschmidt like that?

borgbackup/borgstore#173

[RonnyPfannschmidt]

That's http1

Http2 has very nice properties wrt concurrent use and multiplexin

[ThomasWaldmann]

Yes, we use requests, thus no http2. But it is using 1.1 with a persistent connection.

Maybe we can switch to httpx later for http2 support.

Update: release history of httpx doesn't look that great.

Maybe later when urllib3 has finished it, still "experimental".