Hello, I am writing this because this topic was at first a question I had and I couldn’t find an answer to it, information about it online is scarce and outdated so here I am to share what I have figured out; so

Let’s establish things

  • Remote Machine = The device processing the program/audio and holding the files, streaming them over to the Local Machine
  • Local Machine = The device which initiates the connection to the Remote Machine, hears the audio, interacts with the programs and receives the files requested

What we’ll be using (L/R means Local or Remote respectively)

  • SSH (openSSH) L&R
  • waypipe L&R
  • pipewire, pipewire-pulse and wireplumber L
  • sshfs L
  • Any wlroots based Compositor L
  • Any Terminal Emulator L
  • FUSE L

In my case my compositor of choice will be Labwc, keep in mind all of the components used have a lot of options and you could benefit from checking out whats hot in each of them, I will only cover settings up to things WORKING

First things first install the packages and on the Local Machine make sure you have your sound system running for your User, if you hear audio already you should be okay otherwise review your specific distro documentation on how to start the services

For example on Arch: systemctl start pipewire pipewire-pulse wireplumber --user

Next is to start your Compositor of choice and open up a terminal emulator, you should first make a connection from the Local Machine to the Remote Machine with SSH and your credentials so

For example: ssh -p 7777 -l user 192.168.100.2

Managed to connect to your Remote Machine? Great now we’ll need to do the set-up, we’re going to need to make an environment variable set automatically on each SSH login

This variable has to set-up on SSH LOGIN ONLY as to not disturb the Remote Machine’s local audio playing for when it is used locally, there might be many ways to setup this, in my case I’m gonna add this line to ~/.bash_profile

For example: if ! [ "x${SSH_TTY}" = "x" ]; then export PULSE_SERVER=localhost:4713; fi

This will automatically execute on login, evaluating if it’s an SSH login, and adding the environment variable PULSE_SERVER=localhost:4713 which will tell applications running locally (On the Remote Machine) that the audio socket is the SSH socket we will open, which sends it back to your Local Machine’s audio service (Encrypting it)

When this is done, we can exit the Remote Machine’s shell and everything is basically ready on the Remote Machine

Now on our Local Machine we have to modify our SSH command to create the audio socket we have mentioned prior

For example: ssh -p 7777 -l user -R 4713:localhost:4713 192.168.100.2

The important part is -R 4713:localhost:4713 which is creating the socket on both ends and redirecting through your localhost and onto your audio server’s default listening port (4713), now we just have to make the Audio server listen to your localhost loading it’s Remote sound module

For example: pactl load-module module-native-protocol-tcp listen=127.0.0.1

This needs to be run each time the Local Machine’s audio server restarts, but don’t worry about that adding complexity you won’t need to remember this

Next up we should set-up waypipe, this application allows forwarding both Wayland native applications AND wayland compositors themselves, so if there is an X11 only application you can’t forward through Waypipe, you can start a compositor instead and use it from there (Wine games, to say my use case) just like a Remote Desktop

For example: waypipe --video h264 ssh -p 7777 -l user -R 4713:localhost:4713 192.168.100.2

In my example command, I use hardware accelerated video encoding which greatly increases performance, you may just want to use waypipe alone however which uses default settings, I highly recommend reading waypipe documentation for achieving the best performance for your setup and test it with your application of choice

For example: WLR_RENDERER=gles2 Labwc (executed on the Remote Machine Shell, will open it on your Local Machine’s Compositor)

Finally, for setting up Remote File Access we use sshfs prior to connecting to the Remote Machine, this utility mounts a Remote Filesystem on a local directory through SSH and FUSE using the sftp protocol which is all encrypted

For example: sshfs -p 7777 user@192.168.100.2:/home/user/RemoteDirectory/ /home/user/LocalDirectory/

Nice, now we have it all set up and ready to work, we can finally make it convenient to use, in my case I prefer to run all of this as a script easily accessible on my terminal as a single command that executes the script located on my scripting environment, and we add two more commands that just unload the Audio Server module and the Remote Filesystem mount when we’re done

For an example script:

pactl load-module module-native-protocol-tcp listen=127.0.0.1
sshfs -p 7777 user@192.168.100.2:/home/user/RemoteDirectory/ /home/user/LocalDirectory/
waypipe --video h264 ssh -p 7777 -l user -R 4713:localhost:4713 192.168.100.2
pactl unload-module module-native-protocol-tcp
umount /home/user/LocalDirectory/

Let’s say we create this script and it’s saved in our home folder, we just have to make it executable (chmod +x scriptdir) and run it from our Terminal Emulator

For example: ./Remote\ Machine1.sh

And it will automatically set up everything for us and ask for our Credentials, we have a perfect workspace that imitates that of a remote desktop experience, on Wayland (may not be exclusive to this but that’s what I’m doing here)

Did I miss anything? Have suggestions? Let me know what can be done better and I’ll update this post, thanks for reading and have a good one

  • petrichornetrainfall@piefed.social
    link
    fedilink
    English
    arrow-up
    13
    ·
    10 hours ago

    You have no idea how grateful I am to you right now.

    This has been on my personal project backlog for awhile now. I have run into so many issues and headaches with possible rdp or vnc solutions for my desired use case, especially with wayland being a must have.

    I have recently fell in love with ssh and was planning on looking more into waypipe as a possible route to take as I kept seeing it recommended, but the examples and documentation i found was always “generic” or surface level and didn’t have enough of the pieces I needed to scrape together.

    I’ve been putting it off due to only having surface level knowledge about all the pieces, and you merged like 10 of them in one go.

    My biggest question to you is what is performance like? Like picture quality, audio delay, and latency/responsiveness around mouse and keyboard inputs? How does it compare to using something like an ipkvm, rdp, rust desktop, etc?

    • Coki91@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      9 hours ago

      Right, so as I mentioned in the post, hardware accelerated encoding for waypipe greatly increases performance and that’s the one thing I can tell you makes the night and day of this setup

      In my experience, my two devices are low-end on my local network, one of them 10 years old (The Local PC) on Ethernet and the other (The “Remote” PC) is in 5G Wi-fi, audio is 1:1 I can barely tell if there’s delay, except when it stutters due to the Wi-fi being wi-fi, for video it’s clearly compressed with h264 so colors are a bit off but otherwise it doesn’t struggle either, latency and responsiveness feel right, like I mentioned in the post WINE Games is my use case for this, and playing Games has a high bar for that, maybe at some point my processor gets jacked to 100% due to the games and it struggles but it has never crashed, and it recovers once intensity tones down

      Compared to other systems, I have only used Xrdp and Microsoft’s RDP on Windows; and I don’t feel any different besides the lack of additional features like drag and drop between devices that aren’t a dealbreaker for me

      • petrichornetrainfall@piefed.social
        link
        fedilink
        English
        arrow-up
        1
        ·
        8 hours ago

        I’ll definitely have to take advantage of the hardware acceleration.

        In regards to gaming, what kinds of games have you typically used this for? I’m not thinking about using this for competitive shooters or something like that where every ms counts, but what about games like a souls like where timing a parry or something matters, have you tried anything like that?

        Lacking those features is perfect fine for me, I’m looking more for a remote display and input implementation anyway.

        One last question, have you experimented with any compression or “enhanced” encryption for the ssh connection? If so, has that impacted the responsiveness at all?

        • Coki91@lemmy.worldOP
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          7 hours ago

          There’s actually some issues with games that make the Mouse Cursor stick to the center and turn the camera with movement (like most FPS do) which also happened on Windows to me, the Mouse has like x10 the sensitivity and is hardly controllable, other games that don’t do that (like point and click games where the Cursor is free) don’t present this issue

          Besides that, I have played mostly keyboard centric games, a fighting-game style MMO that I have engaged in both PvE and PvP Gameplay and besides the bullshit moments in PvP where the Wi-fi turned on me, I really can’t complain of performance, haven’t played Souls games OR games using a joystick controller, it’s probably something that needs testing

          If you mean the compression that waypipe offers with -c, I used for the longest time “-c none” thinking it would free my CPU of all overhead possible, that’s probably the right play on a local network environment but I have since some time ago rid of that to see and I again don’t notice a difference, so if you plan to go out and connect back through a private LAN like Tailscale you may want to keep it on the default compression algorithm

  • just_another_person@lemmy.world
    link
    fedilink
    arrow-up
    1
    arrow-down
    18
    ·
    10 hours ago

    If you’re this familiar with all the moving bits of this, why are you asking?

    I’m only saying this because this is obviously someone who read a bunch of stuff about various things that are beyond their technical expertise but have no idea why they would even need such a stack.

    Kind of a crazy thing to ask about here.

    • Coki91@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      12
      ·
      10 hours ago

      The title kinda reads like a question, so you’re not wrong thinking I’m doing so, however that’s more oriented for like search engines to pick up this post, as often users would type straight up looking for help on something, that’s the idea of that, but in the first paragraph I explain that this is not a question post

    • floofloof@lemmy.ca
      link
      fedilink
      arrow-up
      14
      arrow-down
      1
      ·
      10 hours ago

      They’re offering a helpful guide so others don’t have to spend ages figuring it out for themselves. There’s no need to react so negatively.

        • harmbugler@piefed.social
          link
          fedilink
          English
          arrow-up
          18
          arrow-down
          1
          ·
          10 hours ago

          When you’re doing something this specific, sometime documentation that’s anywhere close is rare and is a godsend. OP, thanks for posting.

        • petrichornetrainfall@piefed.social
          link
          fedilink
          English
          arrow-up
          3
          ·
          8 hours ago

          So a lot of documentation in general typically only covers beginner or expert levels, with almost no intermediate.

          It’s always “here’s how to say hello world” or “here is how to extend our implementation at a low level”.

          So it’s nice to have examples of intermediate use cases for power users who need to do more than the most basic examples, but don’t plan on submitting PRs to add or change functionality.

          More info, guides, and examples are always better than less (as long as it’s accurate).