I want a server running nextcloud, immich and others.

I have a N100 mini server with a 2TB external HDD. I want to secure the system against data loss. Hence, I want a backup and redundancy.

  1. Most important question: How do I build everything? Is this a NAS? My naive approach is to buy 3 external HDDs and connect them to the N100 with a USB hub. I assume this is not “the right way” but to use/build a NAS. Do I have to build a separate NAS computer? When I lookup NAS buying, it is a computer with a case for 4 drives, excluding the drives and costs 400 bucks. I am confused because this is incredibly expensive compared to what I already have. What is the additional benefit compared to my setup? Am I cheap?

  2. Regarding redundancy, is RAID still the way to go? At 2 TB, using RAID 5 with 3 drives sounds good. I’d have 4 TB of usable space, much more than I intend to use in the next years, and adding a drive increases the storage by 2 TB, effectively increasing space by 50%.

  3. I have 4 TB usable space, but I won’t reach 2 TB in the next one or two years. I’d use a 2 TB HDD for a local backup via borg. Once my hot storage needs to increase, I replace the backup drive with a larger one and use it to increase the RAID storage. Is one backup sufficient? Or should I keeping multiple versions of the data. Daily, weekly, monthly backups? What is your experience with it?

  4. Another 2 TB HDD for an offsite backup, LUKS encrypted, backed up once a year (that’s the goal for now).

Does that sound good?

  • mrmaplebar@fedia.io
    link
    fedilink
    arrow-up
    3
    ·
    1 day ago

    I think that sounds pretty solid to me. Realistically you should count on having 3x drives for your important data:

    1. The main data drive(s)
    2. Drive(s) for redundancy, mirroring the data drives. (I use btrfs RAID1 for this.).
    3. Offline local hard drive(s) that you keep somewhere relatively safe that you occasionally backup to.
    4. (Optionally) Some kind of offsite backup.

    So if you plan on having 2TB of data, you’ll ideally want 3x 2TB drives. 2 in the PC mirroring eachother, and 1 in a closet or safe that you plug in and backup to a few times per year. (With bonus points if you can get another 2TB of off site or cloud storage to also backup to, in case of catastrophy.)

    As for how you build it, I think it doesn’t matter too much. Its possible to use whatever random spare PC parts you have to make a decent home server, imo. A lot of people on YouTube and Reddit have all kinds of fancy servers in a rack, but an old repurposed desktop can be fine. ( I would probably use new, decent quality drives though.)

  • ImgurRefugee114@reddthat.com
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 day ago

    Others have answered most questions but I just wanted to point out a few things:

    NAS is network attached storage: a separate server which makes ‘shares’/volumes available over the local network. A DAS is ‘direct attached storage’ which plugs directly into a machine. Since you have a server, a NAS is probably the right route. https://www.techradar.com/features/das-vs-nas-what-is-the-difference

    Good that you mentioned both backups and redundancy as they’re not the same. https://www.tencentcloud.com/techpedia/108368

    Some people want a dedicated machine for being a NAS, while others want to make use of the hardware by making it pull double-duty as a server. I have an old PC I loaded up with drives and installed truenas on: I made zfs pools (opposed to raid) and exposed shares to the network. I can set up virtual machines or use “plugins” / jails for hosting other services like immich etc. E.g. https://www.truenas.com/docs/solutions/integrations/nextcloud/

    Regarding backup versioning, modern filesystems like zfs and brfs have snapshot features. Regarding “one backup”, it depends how important it is to you. https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

  • anamethatisnt@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    11
    ·
    2 days ago

    If your N100 mini server has an ITX motherboard with several SATA ports then changing the case to something like a Jonsbo N2 might be a solution that works better than USB.

  • Mylk@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    1
    ·
    2 days ago

    Lot to unpack, just a quick answer:

    1. Don’t do that, forget the USB hub it’s unreliable, if you absolutely want to go with the USB route get something like a HDD multi bay enclosure: https://sabrent.com/collections/hard-drive-accessories/products/ds-sc4b But if you don’t want to fiddle with things get a ready made solution (qnap, synology, ugreen, etc) you get the hardware and the software, just insert the disks and you are ready to go.
    2. If you are doing DIY look into ZFS, or use the solution from the software you are using, Unraid for example.
    3. The usual advice is to do the 3-2-1 backup (three copies of your data across two different storage types with one off-site), but entirely depends on you and your data, keep at least a separate copy of your most important things.
    4. Sounds good.
  • MentalEdge@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    5
    ·
    edit-2
    2 days ago
    1. There is no “special” benefit to a pre-built NAS. They have convenient software but there is nothing exceptional about them. They’re just computers with storage drive slots. Using a bunch of external drives via a USB hub would be fine. But is that your only expansion option on the system you have? Access speeds via USB, especially if using a hub, won’t be ideal. It’ll certainly work, though. You can also get enclosures to put full size HDDs in, which can connect to an existing system.

    2. RAID is still the way to go, but since you don’t need much storage, I’d start with RAID 1, not 5. 5 will require a rebuild with a new drive if something goes wrong, while RAID 1 will work with 2 drives and give you complete mirroring. Since you intend to have a “local” backup copy anyway, why not just skip that and use RAID 1? It’s literally the same thing, except it’ll actually provide uptime in case of failure, unlike a backup drive or raid 5.

    3. So RAID 5 plus a local backup, plus another offsite? This is overkill IMO. (Not the offsite backup that’s good. But raid+local copy. Just use two drives and mirror them using whatever you prefer.) In your place, I think I’d go with BTRFS in raid1c2 mode. This is like raid1, in that with two drives, you only get the capacity of one drive. But, the “c2” means that each data block is mirrored to two drives. With more than two drives, you can expand storage. (With three 2TB drives you’d get 3TB) You don’t get as much available storage as with raid5, but you get expandability, which you normally don’t with raid1. And you get uptime in case of failure without an array rebuild (though for this you must mount the volume with the “degraded” option, unlike actual raid using mdadm). You also get filesystem snapshots.

    4. You intend to do this manually? That is fine. My current solution is a second NAS at my dad’s home, to which my system is backed up daily using Kopia. Kopia deduplicates and compresses the backups, efficiently keeping versions up to two years back. The simplest version of this would be a router that can host an FTP server using an external drive in its usb port. This way you could automate off-site backup and have it happen more frequently. Asus routers can do this, and even come with free dynamic DNS and automatic https with letsenrypt. You literally just plug it into WAN somewhere, and you’ll be able to back up to it over the internet.


    Finally, just some mentions.

    MDADM, is what you’d use to create a software RAID array.

    BTRFS has built-in multi-device storage, of which only single, raid0, and raid1 are stable. Do not use the raid5 and 6 modes. While named raid, the modes differ from actual raid. BTRFS is able to convert from one mode to another, and can add drives in any mode (though will need to “balance” the drives after changes, to make additional capacity available). It is also able to evict drives. It will not auto-mount a volume after drive failure, and requires the “degraded” option be added.

    Mergerfs can be used to merge filesystems to expand storage non-destructively. It is able to arbitrarily combine volumes of any type, to combine their capacity. This way, it can for example be used to expand a raid1 array by combining it with a single disk, or another raid1 array, or whatever else. This can be done temporarily, as the combined volume can also be disassembled non-destructively, with each file simply remaining on whatever drive they were on.

  • tburkhol@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    2 days ago

    Depending on the board in your mini-server, you may have enough SATA ports to plug in directly. I have a system similar to what you’re describing (N100 with 4x 2TB HDDs with 1.5TB data): 2 of those drives are set up in RAID1 (mirror), and once a month, I plug in one of the spares, rsync the array to it, and unplug it. Every 3 months or so, I swap the offline drive with an offsite drive. I used to use a USB dock for the offline drive, but I got a 3-bay hot-swap enclosure to make the whole process faster and easier.

    The server shares the array via NFS and SMB, and it is absolutely a NAS for all my other systems.

    If you expect to exceed 2TB data within 2 years, then you’ll need to replace all 4 of those 2TB drives in 2 years. You might, today, get a pair of 4 TB drives and one 2TB, use the 4TB as your main storage, the 2TBs as rotating backups, and wait until you actually outgrow 2TB to upgrade the backups.

  • Onomatopoeia@lemmy.cafe
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    8
    ·
    edit-2
    1 day ago

    RAID isn’t data redundancy, it’s an array of drives combined to form a single logical storage pool. It solves the problem of needing a single storage pool larger than the available drives. As such, it’s very sensitive to loss of a single drive.

    At your storage size requirements (2 TB), RAID is unnecessary today.

    Edit: Let me say it again for you downvoters-RAID is NOT data redundancy.

    There is only ONE copy of your data in RAID (excepting mirroring). It’s why RAID now has double parity and hot spare drive capability.

    RAID is for creating a single pool that’s larger than available drive size.

    Go ahead and downvote in ignorance, and learn about data redundancy when your RAID fails.

    RAID is NOT data redundancy - it’s DRIVE redundancy.

    Take it from the source https://www2.eecs.berkeley.edu/Pubs/TechRpts/1987/CSD-87-391.html

    • talkingpumpkin@lemmy.world
      link
      fedilink
      English
      arrow-up
      15
      ·
      2 days ago

      RAID (except RAID0) is data redundancy, it just isn’t backup (ie. it doesn’t help if you accidentally delete stuff, or if some bug corrupts it, or if you drop the computer while moving it).

    • ExcessShiv@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      9
      ·
      2 days ago

      You’re using redundancy and backup synonymously, but they’re not. Raid 1+ absolutely provides redundancy, you are 100% wrong in saying that it doesn’t, because it provides a failover system that prevents operational interruption if a drive fails.

      • Onomatopoeia@lemmy.cafe
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        3
        ·
        2 days ago

        That’s not data redundancy - there’s still only one copy of your data.

        Those are mitigations against loss of data due to loss of parity.

        There’s still only ONE copy of your data.

        • Courant d'air 🍃@jlai.lu
          link
          fedilink
          English
          arrow-up
          5
          ·
          2 days ago

          I think you’re confusing backup and redundancy. While I totally agree RAID1+ should not be considered backup, it absolutely is redundancy, as the same data is present on at least two disks (either on the form of the exact same data or something that can be used to rebuild the missing data).