Synology Shrink a SHR Volume and remove disks from a Synology

I wanted to try this since years... I finally did it successfully in a virtual Synology with an array of 5 disks in a single volume using btrfs (See here for VM and btfrs).

Click to Read More

This is a summary of my attempts. It is based on various posts found on the web:

  • https://superuser.com/questions/834100/shrink-raid-by-removing-a-disk
  • https://unix.stackexchange.com/questions/67702/how-to-reduce-volume-group-size-in-lvm#67707
  • https://blog.reboost.net/manually-removing-a-disk-from-a-synology-raid/

Notice that xfs File System only supports "extend" and not "reduce". So, I only tried with etx4 and btrfs.

Although I used a DSM with a video station, a photo station, the web station and wordpress, this was not a "real case" (most of the data were still on the first sectors of the disks). So, I would really not recommend to do this on a real NAS !!!

 

Open a SSH console and enter root mode (sudo -i) to execute the commands here after.

To identify which process is accessing a volume, we possibly need lsof (only if umount fails)

Install OPKG : See https://www.beatificabytes.be/use-opkg-instead-of-ipkg-on-synology/

Install lsof: /opt/bin/opkg install lsof

Layout of the Physical Volume/Volume Group/ Logical Volumes :

Find the Filesytem of /volume1:

df -h

Filesystem      Size  Used Avail Use% Mounted on
/dev/md0        2.3G 1016M  1.2G  46% /
none            2.0G     0  2.0G   0% /dev
/tmp            2.0G  536K  2.0G   1% /tmp
/run            2.0G  3.2M  2.0G   1% /run
/dev/shm        2.0G  4.0K  2.0G   1% /dev/shm
none            4.0K     0  4.0K   0% /sys/fs/cgroup
cgmfs           100K     0  100K   0% /run/cgmanager/fs
/dev/vg1000/lv   14G  3.2G  9.9G  25% /volume1

  • Show Physical Volume: pvdisplay
  • Show Volume Group: vgdisplay
  • Show Logical Volume: lvdisplay
  • Show Disks: fdisk -l

Check if one can umount the volume1 :

umount /dev/vg1000/lv

umount: /volume1: target is busy
       (In some cases useful info about processes that
        use the device is found by lsof(8) or fuser(1).)

Stop all services :

synopkg onoffall stop

/usr/syno/etc.defaults/rc.sysv/S80samba.sh stop

/usr/syno/etc.defaults/rc.sysv/S83nfsd.sh stop

/usr/syno/etc.defaults/rc.sysv/pgsql.sh stop

/usr/syno/etc.defaults/rc.sysv/synomount.sh stop

check which daemons are still using volume1 :

/opt/bin/lsof | grep volume1

COMMAND    PID  TID     USER  FD        TYPE                          DEVICE SIZE/OFF       NODE NAME
s2s_daemo 10868         root    8u      REG               0,30    11264        608 /volume1/@S2S/event.sqlite
synologan  8368         root  3u        REG               0,30       3072        654 /volume1/@database/synologan/alert.sqlite
synoindex  8570         root  mem       REG               0,28                  7510 /volume1/@appstore/PhotoStation/usr/lib/libprotobuf-lite.so (path dev=0,30)
synoindex  8570         root  mem       REG               0,28                 30143 /volume1/@appstore/VideoStation/lib/libdtvrpc.so (path dev=0,30)
lsof       8585         root  txt       REG               0,30     147560      32161 /volume1/@entware-ng/opt/bin/lsof

Kill those daemons :

initctl list | grep synoindex

synoindexcheckindexbit stop/waiting
synoindexd start/running, process 11993

killall synoindexd

initctl list | grep synologan

synologand start/running, process 8368

killall synologand

This is not working… synologand  restart immediately ☹

chmod u-x /usr/syno/sbin/synologand

killall synologand 

killall s2s_daemon

Shrink the ext2 or ext4 Filesystem :

Only do the steps here under if using ext2 or ext4 instead of btfrs

Resize the File System :

umount -d /dev/vg1000/lv

e2fsck -f /dev/vg1000/lv

e2fsck 1.42.6 (21-Sep-2012)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
/lost+found not found. Create<y>? yes
Pass 4: Checking reference counts
Pass 5: Checking group summary information

1.42.6-23739: ***** FILE SYSTEM WAS MODIFIED *****
1.42.6-23739: 30742/889440 files (0.7% non-contiguous), 734048/3553280 blocks

resize2fs -M /dev/vg1000/lv

resize2fs 1.42.6 (21-Sep-2012)
Resizing the filesystem on /dev/vg1000/lv to 702838 (4k) blocks.
The filesystem on /dev/vg1000/lv is now 702838 blocks long.

If you see the error message here under, you possibly have another file system than ext2 or ext4 (E.g.: btfrs, ...):

resize2fs: Bad magic number in super-block while trying to open /dev/vg1000/lv
Couldn't find valid filesystem superblock.

If you have this error message although having ext2 or ext4, try: lvm lvchange --refresh /dev/vg1000/lv

Check the results :

mount /dev/vg1000/lv /volume1
df -h

Filesystem Size Used Avail Use% Mounted on
/dev/md0 2.3G 968M 1.3G 44% /
none 2.0G 0 2.0G 0% /dev
/tmp 2.0G 512K 2.0G 1% /tmp
/run 2.0G 3.0M 2.0G 1% /run
/dev/shm 2.0G 4.0K 2.0G 1% /dev/shm
none 4.0K 0 4.0K 0% /sys/fs/cgroup
cgmfs 100K 0 100K 0% /run/cgmanager/fs
/dev/vg1000/lv 2.6G 2.5G 0 100% /volume1

Resize the Logical Volume :

Resize the logical volume a bit larger than the file system (See the outcome of df -h above)

umount /dev/vg1000/lv
lvm lvreduce -L 2.7G /dev/vg1000/lv

Rounding size to boundary between physical extents: 2.70 GiB
WARNING: Reducing active logical volume to 2.70 GiB
THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce volume_1? [y/n]: y
Size of logical volume vg1/volume_1 changed from 13.55 GiB (3470 extents) to 2.70 GiB (692 extents).
Logical volume volume_1 successfully resized.

 

Shrink a BTRFS File System :

Only do the steps here under if using btfrs

Resize the File System :

btrfs filesystem resize 2.7G /volume1

You can restore max size if required: btrfs filesystem resize max /volume1

df -h

Filesystem      Size  Used Avail Use% Mounted on
/dev/vg1000/lv  2.6G 2.5G  1.4G  71% /volume1

Resize the Logical Volume:

umount -d /dev/vg1000/lv

lvm lvreduce -L 2.7G /dev/vg1000/lv

 

Next steps :

The steps here under are both for ext2/ext4 and btfrs

Resize the Physical Volume :

Look for the device and if blocks must be moved:

pvdisplay -C

PV VG Fmt Attr PSize PFree
/dev/md2 vg1 lvm2 a-- 13.57g 10.85g

pvs -v --segments /dev/md2

    Using physical volume(s) on command line.
   Wiping cache of LVM-capable devices
PV         VG     Fmt  Attr PSize  PFree   Start SSize LV   Start Type   PE Ranges
/dev/md2   vg1000 lvm2 a--  14.00g 440.00m     0  3473 lv       0 linear /dev/md2:0-3472
/dev/md2   vg1000 lvm2 a--  14.00g 440.00m  3473   110          0 free

If there is a bloc trailing after the "free" part (I had not), use:

lvm pvmove --alloc anywhere /dev/md2:xxx-xxx

 

Resize the device a bit larger than the logical volume. If you don't use a sufficient size, you will get an error message:

pvresize --setphysicalvolumesize 2.7G /dev/md2

/dev/md2: cannot resize to 691 extents as 695 are allocated.
0 physical volume(s) resized / 1 physical volume(s) not resized

pvresize --setphysicalvolumesize 2.8G /dev/md2

Physical volume "/dev/md2" changed
1 physical volume(s) resized / 0 physical volume(s) not resized

 

Resize the array to use less disks:

Reduce the array to use 3 disks (-n3)

mdadm --grow -n3 /dev/md2

mdadm: this change will reduce the size of the array.
      use --grow --array-size first to truncate array.
      e.g. mdadm --grow /dev/md2 --array-size 7114624

You have first to reduce the array size.

mdadm --grow /dev/md2 --array-size 7114624

mdadm --grow -n3 /dev/md2 --backup-file /root/mdam.md2.backup &

 

Monitor the progress of the resizing with:

cat /proc/mdstat

Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [raidF1]md2 : active raid5 sdak5[4] sdaj5[3] sdai5[2] sdah5[1] sdag5[0]7114624 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU][======>..............] reshape = 31.7% (1129600/3557312) finish=0.3min speed=112960K/sec

md1 : active raid1 sdak2[4] sdaj2[3] sdai2[2] sdah2[1] sdag2[0]2097088 blocks [12/5] [UUUUU_______]
md0 : active raid1 sdak1[4] sdaj1[3] sdag1[0] sdah1[1] sdai1[2]2490176 blocks [12/5] [UUUUU_______]
unused devices: <none>

This can take a lot of time, but you can continue (I did wait ;) )

Finalize

pvresize /dev/md2

Physical volume "/dev/md2" changed
1 physical volume(s) resized / 0 physical volume(s) not resized

lvextend -l 100%FREE /dev/vg1000/lv

Size of logical volume vg1/volume_1 changed from 2.70 GiB (692 extents) to 4.07 GiB (1041 extents).
Logical volume volume_1 successfully resized.

for btrfs:

btrfs filesystem resize max /volume1

for ext2/ext4:

e2fsck -f /dev/vg1000/lv

e2fsck 1.42.6 (21-Sep-2012)
Pass 1: Checking inodes, blocks, and sizes

Running additional passes to resolve blocks claimed by more than one inode...
Pass 1B: Rescanning for multiply-claimed blocks
Multiply-claimed block(s) in inode 13: 9221
Pass 1C: Scanning directories for inodes with multiply-claimed blocks
Pass 1D: Reconciling multiply-claimed blocks
(There are 1 inodes containing multiply-claimed blocks.)

File /@tmp (inode #13, mod time Sat May 23 23:00:14 2020)
has 1 multiply-claimed block(s), shared with 0 file(s):
Multiply-claimed blocks already reassigned or cloned.

Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
1.42.6-23739: 30742/179520 files (0.7% non-contiguous), 685397/720896 blocks

Hope that no nodes are corrupted. Otherwise... well... accept to fix them.

resize2fs /dev/vg1000/lv

resize2fs 1.42.6 (21-Sep-2012)
Resizing the filesystem on /dev/vg1/volume_1 to 1065984 (4k) blocks.
The filesystem on /dev/vg1/volume_1 is now 1065984 blocks long.

mount /dev/vg1000/lv /volume1

Restart services:

/usr/syno/etc.defaults/rc.sysv/S80samba.sh start

/usr/syno/etc.defaults/rc.sysv/S83nfsd.sh start

/usr/syno/etc.defaults/rc.sysv/pgsql.sh start

/usr/syno/etc.defaults/rc.sysv/synomount.sh start

 

synopkg onoffall start

 

Remove the new  spare disks from the volume

mdadm --detail --scan

ARRAY /dev/md0 metadata=0.90 UUID=3b122d95:7efea8ff:3017a5a8:c86610be

ARRAY /dev/md1 metadata=0.90 UUID=bd288153:d00708bf:3017a5a8:c86610be

ARRAY /dev/md2 metadata=1.2 spares=2 name=DS3617_62:2 UUID=875ad2d6:956306b7:8c7ba96b:4287f6e6


mdadm --detail /dev/md2

/dev/md2:
       Version : 1.2
Creation Time : Sat May 23 14:00:02 2020
    Raid Level : raid5
    Array Size : 7114624 (6.79 GiB 7.29 GB)
Used Dev Size : 3557312 (3.39 GiB 3.64 GB)
  Raid Devices : 3
Total Devices : 5
   Persistence : Superblock is persistent

    Update Time : Sat May 23 18:54:30 2020
         State : clean
Active Devices : 3
Working Devices : 5
Failed Devices : 0
Spare Devices : 2

         Layout : left-symmetric
    Chunk Size : 64K

           Name : DS3617_62:2  (local to host DS3617_62)
          UUID : 875ad2d6:956306b7:8c7ba96b:4287f6e6
        Events : 56

    Number   Major   Minor   RaidDevice State
      0       8        5        0      active sync   /dev/sda5
      1       8       37        1      active sync   /dev/sdc5
      2       8       53        2      active sync   /dev/sdd5
      3       8       69        -      spare   /dev/sde5
      4       8       85        -      spare   /dev/sdf5

Here above, we see that sde5 and sdf5 are unused

 

mdadm --fail /dev/md2 /dev/sde5

mdadm: set /dev/sde5 faulty in /dev/md2

mdadm --fail /dev/md2 /dev/sdf5

mdadm: set /dev/sdf5 faulty in /dev/md2

mdadm --remove /dev/md2 /dev/sde5

mdadm: hot removed /dev/sde5 from /dev/md2

mdadm --remove /dev/md2 /dev/sdf5

mdadm: hot removed /dev/sdf5 from /dev/md2

 

Et voilà.

Synology Add Support for SHR on Synology DS3617xs

I wanted to test a procedure to shrink a SHR volume on a virtual Synology DS3617. Unfortunately, SHR is not an available option on the high end models of Synology. But it can be enabled easily ;)

Click to Read More

DS3617xs is a model where SHR is not enabled:

0KenZdg.png

To enable it:

  1. Open a SSH console and enter root mode.
  2. Edit the file  /etc.defaults/synoinfo.conf
  3. At the end, you should find: supportraidgroup="yes"
  4. Comment that line with a #
  5. Next to it, add a new line with: support_syno_hybrid_raid="yes"
  6. Reboot your Synology

You can use "vi" to edit the file: sudo vi /etc.defaults/synoinfo.conf,

  • Press i or the insert key to make your modifications, then "Esc" to end the modifications.
  • After that type :x! to save and exit. Ot :q! to exit without saving.

Et voilà, your DS3617xs offers now the option for SHR

Tips VMWare Workstation not compatible with Windows 10

Starting a VM this morning, I got a new message "VMware Workstation and Device/Credential Guard are not compatible". To be able to run my VM, I had to disable this Credential Guard.

Click to Read More

Credential Guard uses virtualization to store credentials in protected containers separate from the OS. As a result, the information Credential Guard protects is safe even if malware or some other malicious attack penetrates an organization's network.

But it prevents VMWare to run VM's.

To disable it, copy this script in a file named "disableCG.ps1" in your personal home folder C:\Users\<Your Name>\

Run next a PowerShell as Administrator:

Pay Attention that this will reboot automatically your PC !!

In this PowerShell, execute: cd ~

Next execute : .\disableCG.ps1 -Disable -AutoReboot

You will see errors displayed in Red in the PowerShell. This won't be a problem. And after a minute, the PC will reboot.

During the reboot, you will be prompted to confirm that you really want to disable the Windows Defender Credential Guard. Press the Windows Key or F3 to confirm.

 

Other options : https://windowsreport.com/disable-credential-guard-windows-10/

Ex. Execute in a PowerShell run as Administrator: bcdedit /set hypervisorlaunchtype off

 

I also had to remove "Guarded Host" and "Hyper -V" via the Control Panel > Programs > Turn Windows features on or off

But now, Docker does not run anymore on my PC as it needs Hyper-V :(

SmartHome Philips Hue lights Bulbs and Accessories become unreachable

Regularly, my Philips Hue lights and accessories do not respond anymore. Looking into the Philips Hue App, they appear as "Unreachable" (no link). The solution was to avoid WiFi interference!

Click to Read More

In French, the message displayed by the App, for the Lights and accessories, is "impossible d'établir le lien".

If your Philips Hue lights aren't responding, maybe it's time to ...

 

Notice that each time, a few bulbs and accessories were still connected and the App was perfectly able to connect onto the Bridge, both via Internet and via my local Network. Notice also that I have many Philips Hue Light Bulbs, Sensors and Dimmer Switches in the house (less than 50, which is the theoretical maximum for a Philips Hue Bridge) as well as a few Tradfri Bulbs in the corridors.

I really spent hours to investigate this issue which occurred once every few months ...

  • Unplug the power supply of the Philips Bridge for 30 sec.
  • Switch off all the lights for 30 sec.
  • Remove the Bulbs from their sockets.
  • Press the 4 buttons of the remotes (Philips Hue Dimmer Switch) all together for a few seconds (until the led start to blink red/green).
  • Press both the On and Off buttons of the remote to reset the Bulbs.
  • Turn on/off the lights quickly several times (and do an incantation for Apollo).
  • Delete and Re-add again the lights and accessories within the App.
  • Go through the whole FAQ of Philips Hue.
  • Drink a bottle of wine and cry alone in my sofa.
  • etc...

I was always finally switching off all electric devices which were possibly disrupting the Zigbee signals : my routers, my wifi access points, my other Bridges (Tahoma Somfy, Z-Wave, ...) ... and even powering off the whole house... This extreme action was solving the problem ! But impossible to conclude what was the precise root cause.

Today, the problem occurred again. After various unsuccessful attempts (those mentioned above),  I read on a blog that the quality of the ethernet connection between the Philips Hue Bridge and Internet matters ?!? And I realized that my Bridge was plugged into a Devolo Powerline Adapter (Ethernet via Power Sockets). I also remember that in the past, unplugging and replugging that Devolo Powerline Adapter was an action I did and which solved the problem (but I actually did this to restart the various Bridges and WiFi access points connected onto it).

So, I moved the Philips Hue devices from that Adapter into the main Ethernet Router and boom, it worked !

Et voilà ? Really not sure. The ethernet connection is only for the Bridge to have access on a Wifi Access Point and be able to connect with the mobile App... The bulbs and the Bridge are connected via Zigbee. So, there  must be another reason hidden behind this (fake) "solution".

Note that the Phillips Hue Light Bulbs, even though they are Zigbee, do not act as repeaters for anything except other Light Bulbs on the same Hue bridge if they are connected to the Hue bridge (They use actually a "Zigbee Light Link" protocol instead of the regular Zigbee).

As I noticed that the disconnected Bulbs are often the Tradfri ones and the Philips Hue far from the Bridge, I thought that the problem was possibly more with the Tradfri light bulbs not receiving/repeating properly the signal for the Philips Hue light bulbs ? Two possible options:

  • Replace the cheap Tradfri light bulbs with expensive Philips Hue light bulbs. I will do so soon or later...
  • Investigate why receiving/repeating the ZigBee signal could fail... I can start with this.

It's often mentioned on the web that ZigBee can suffer for WiFi interferences. It shares the same 2.4GHz wireless band as WiFi routers and overlaps by default with the Wifi channel 11 (https://statusq.org/archives/2018/01/09/8435/). And indeed, moving the Philips Hue Bridge or Turning On/Off all Wifi Devices in the house used to solve, at least temporarily, the problem.

Ex.: Zigbee channels 21 to 24 overlaps with WiFi channel 11 as illustrated bellow. 

A first obvious solution  is to disable the WiFi 2.4GHz band and use only the 5GHz band for the WiFi Devices at home. But not all of them support 5GHz...

Another solution is to change the channel used by the Philips Hue Bridge  to use a ZigBee channel having less conflict with your WiFi (All lights must be powered on). Do the change via Philips Hue App's settings > Hue Bridges > touch the "i" of the Bridge to update > ZibBee channel change > Change Channel.

It will take about 30 seconds, but the Hue system will change the channel to another one automatically. Once done, the lights should all come back online (And indeed, this did the trick once for me too, without doing any other action). If they don’t, turn them on and off again at the main light switch/plug socket. Possible Zigbee channels are 11, 15, 20 and 25

Philips Hue Zigbee channel

Also you can try to move the Philips Hue Bridge to another location, not too close of the WiFi access points and in central location regarding the various ZigBee devices. This is what I did by connecting the Bridge directly on the Router (So the solution was not the improvement of the ethernet connection quality, but most probably a lower WiFi interference next to that Router).

Notice that this won't update the channel used by the Tradfri lights! As explained on Reddit by Erik1971, you can try, after changing the Hue System channel, to power cycle the Tradfri lights (make sure they are not powered for at least 1 minute). This should trigger the Tradfri lights to scan over the channels to join the network again. Best way to do this is to power off all Tradfri lights and keep the Hue Lights and Bridge powered and then one by one power on the Tradfri lights (to prevent that the Tradfri Lights see the other Tradfri lights on the wrong channel and rejoin on the wrong channel).

So, for the time being, my conclusion is that bulbs and accessories became unreachable due to (many?) new devices connecting on the WiFi or WiFi Access Points changing automatically their channels (This can be checked with an App like Wifi Analyzer).

Synology Backup Synology to Unraid

The easiest way to backup a Synology NAS to Unraid Server is to use Hyper Backup on Synology and rsync on Unraid.

Click to Read More

First, enable rsync on your Unraid Server. It is preinstalled but not runing as a daemon.

Create a file /boot/custom/etc/rsyncd.conf with the following content:

uid             = root
gid             = root
use chroot      = no
max connections = 4
pid file        = /var/run/rsyncd.pid
timeout         = 600

[backups]
    path = /mnt/user/backups
    comment = Backups
    read only = FALSE

Here above:

The name "backups" between brackets will be visible as "backup module" from the Synology. You can create several blocks like this one.

The "path" must exist on your Unraid server.

 

Next, create a file /boot/custom/etc/rc.d/S20-init.rsyncd with the following content:

#!/bin/bash

if ! grep ^rsync /etc/inetd.conf > /dev/null ; then
cat <<-EOF >> /etc/inetd.conf
rsync   stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/bin/rsync --daemon
EOF
read PID < /var/run/inetd.pid
kill -1 ${PID}
fi

cp /boot/custom/etc/rsyncd.conf /etc/rsyncd.conf

Finally, add the following line in the file /boot/config/go :

#!/bin/bash
/boot/custom/etc/rc.d/S20-init.rsyncd

 

Now, either reboot or execute: bash /boot/custom/etc/rc.d/S20-init.rsyncd

 

Go now on your Synology and open "Hyper Backup" to Create a new Data Backup Task:

Select rsync as Backup Destination:

And Create the backup Task with "rsync-compatible server" as Server Type: