Month: May 2020

  • Shrink a SHR Volume and remove disks from a Synology

    I wanted to try this since years… I finally did it successfully in a virtual Synology with an array of 5 disks in a single volume using btrfs (See here for VM and btfrs).

    Click to Read More

    This is a summary of my attempts. It is based on various posts found on the web:

    • https://superuser.com/questions/834100/shrink-raid-by-removing-a-disk
    • https://unix.stackexchange.com/questions/67702/how-to-reduce-volume-group-size-in-lvm#67707
    • https://blog.reboost.net/manually-removing-a-disk-from-a-synology-raid/

    Notice that xfs File System only supports “extend” and not “reduce”. So, I only tried with etx4 and btrfs.

    Although I used a DSM with a video station, a photo station, the web station and wordpress, this was not a “real case” (most of the data were still on the first sectors of the disks). So, I would really not recommend to do this on a real NAS !!!

     

    Open a SSH console and enter root mode (sudo -i) to execute the commands here after.

    To identify which process is accessing a volume, we possibly need lsof (only if umount fails)

    Install OPKG : See https://www.beatificabytes.be/use-opkg-instead-of-ipkg-on-synology/

    Install lsof: /opt/bin/opkg install lsof

    Layout of the Physical Volume/Volume Group/ Logical Volumes :

    Find the Filesytem of /volume1:

    df -h

    Filesystem      Size  Used Avail Use% Mounted on
    /dev/md0        2.3G 1016M  1.2G  46% /
    none            2.0G     0  2.0G   0% /dev
    /tmp            2.0G  536K  2.0G   1% /tmp
    /run            2.0G  3.2M  2.0G   1% /run
    /dev/shm        2.0G  4.0K  2.0G   1% /dev/shm
    none            4.0K     0  4.0K   0% /sys/fs/cgroup
    cgmfs           100K     0  100K   0% /run/cgmanager/fs
    /dev/vg1000/lv   14G  3.2G  9.9G  25% /volume1

    • Show Physical Volume: pvdisplay
    • Show Volume Group: vgdisplay
    • Show Logical Volume: lvdisplay
    • Show Disks: fdisk -l

    Check if one can umount the volume1 :

    umount /dev/vg1000/lv

    umount: /volume1: target is busy
           (In some cases useful info about processes that
            use the device is found by lsof(8) or fuser(1).)

    Stop all services :

    synopkg onoffall stop

    /usr/syno/etc.defaults/rc.sysv/S80samba.sh stop

    /usr/syno/etc.defaults/rc.sysv/S83nfsd.sh stop

    /usr/syno/etc.defaults/rc.sysv/pgsql.sh stop

    /usr/syno/etc.defaults/rc.sysv/synomount.sh stop

    check which daemons are still using volume1 :

    /opt/bin/lsof | grep volume1

    COMMAND    PID  TID     USER  FD        TYPE                          DEVICE SIZE/OFF       NODE NAME
    s2s_daemo 10868         root    8u      REG               0,30    11264        608 /volume1/@S2S/event.sqlite
    synologan  8368         root  3u        REG               0,30       3072        654 /volume1/@database/synologan/alert.sqlite
    synoindex  8570         root  mem       REG               0,28                  7510 /volume1/@appstore/PhotoStation/usr/lib/libprotobuf-lite.so (path dev=0,30)
    synoindex  8570         root  mem       REG               0,28                 30143 /volume1/@appstore/VideoStation/lib/libdtvrpc.so (path dev=0,30)
    lsof       8585         root  txt       REG               0,30     147560      32161 /volume1/@entware-ng/opt/bin/lsof

    Kill those daemons :

    initctl list | grep synoindex

    synoindexcheckindexbit stop/waiting
    synoindexd start/running, process 11993

    killall synoindexd

    initctl list | grep synologan

    synologand start/running, process 8368

    killall synologand

    This is not working… synologand  restart immediately ☹

    chmod u-x /usr/syno/sbin/synologand

    killall synologand 

    killall s2s_daemon

    Shrink the ext2 or ext4 Filesystem :

    Only do the steps here under if using ext2 or ext4 instead of btfrs

    Resize the File System :

    umount -d /dev/vg1000/lv

    e2fsck -C 0 -f /dev/vg1000/lv

    e2fsck 1.42.6 (21-Sep-2012)
    Pass 1: Checking inodes, blocks, and sizes
    Pass 2: Checking directory structure
    Pass 3: Checking directory connectivity
    /lost+found not found. Create<y>? yes
    Pass 4: Checking reference counts
    Pass 5: Checking group summary information

    1.42.6-23739: ***** FILE SYSTEM WAS MODIFIED *****
    1.42.6-23739: 30742/889440 files (0.7% non-contiguous), 734048/3553280 blocks

    NB. If you want to stop e2fsck : killall -USR2 e2fsck
    NB. If you want to get a progress from e2fsck : killall -USR1 e2fsck

    resize2fs -p -M /dev/vg1000/lv

    resize2fs 1.42.6 (21-Sep-2012)
    Resizing the filesystem on /dev/vg1000/lv to 702838 (4k) blocks.
    The filesystem on /dev/vg1000/lv is now 702838 blocks long.

    If you see the error message here under, you possibly have another file system than ext2 or ext4 (E.g.: btfrs, …):

    resize2fs: Bad magic number in super-block while trying to open /dev/vg1000/lv
    Couldn't find valid filesystem superblock.

    If you have this error message although having ext2 or ext4, try: lvm lvchange –refresh /dev/vg1000/lv

    Check the results :

    mount /dev/vg1000/lv /volume1
    df -h

    Filesystem Size Used Avail Use% Mounted on
    /dev/md0 2.3G 968M 1.3G 44% /
    none 2.0G 0 2.0G 0% /dev
    /tmp 2.0G 512K 2.0G 1% /tmp
    /run 2.0G 3.0M 2.0G 1% /run
    /dev/shm 2.0G 4.0K 2.0G 1% /dev/shm
    none 4.0K 0 4.0K 0% /sys/fs/cgroup
    cgmfs 100K 0 100K 0% /run/cgmanager/fs
    /dev/vg1000/lv 2.6G 2.5G 0 100% /volume1

    Resize the Logical Volume :

    Resize the logical volume a bit larger than the file system (See the outcome of df -h above)

    umount /dev/vg1000/lv
    lvm lvreduce -L 2.7G /dev/vg1000/lv

    Rounding size to boundary between physical extents: 2.70 GiB
    WARNING: Reducing active logical volume to 2.70 GiB
    THIS MAY DESTROY YOUR DATA (filesystem etc.)
    Do you really want to reduce volume_1? [y/n]: y
    Size of logical volume vg1/volume_1 changed from 13.55 GiB (3470 extents) to 2.70 GiB (692 extents).
    Logical volume volume_1 successfully resized.

    NB: to get a progress, use : LV=/dev/vg1000/lv; echo `lvdisplay -v $LV | grep current | wc -l` `lvdisplay -v $LV | grep stale | wc -l` | awk ‘{printf ( “%3d percent Complete \n”, 100-$2/$1*100) }’

    Shrink a BTRFS File System :

    Only do the steps here under if using btfrs

    Resize the File System :

    btrfs filesystem resize 2.7G /volume1

    You can restore max size if required: btrfs filesystem resize max /volume1

    df -h

    Filesystem      Size  Used Avail Use% Mounted on
    /dev/vg1000/lv  2.6G 2.5G  1.4G  71% /volume1

    Resize the Logical Volume:

    umount -d /dev/vg1000/lv

    lvm lvreduce -L 2.7G /dev/vg1000/lv

     

    Next steps :

    The steps here under are both for ext2/ext4 and btfrs

    Resize the Physical Volume :

    Look for the device and if blocks must be moved:

    pvdisplay -C

    PV VG Fmt Attr PSize PFree
    /dev/md2 vg1 lvm2 a-- 13.57g 10.85g

    pvs -v –segments /dev/md2

        Using physical volume(s) on command line.
       Wiping cache of LVM-capable devices
    PV         VG     Fmt  Attr PSize  PFree   Start SSize LV   Start Type   PE Ranges
    /dev/md2   vg1000 lvm2 a--  14.00g 440.00m     0  3473 lv       0 linear /dev/md2:0-3472
    /dev/md2   vg1000 lvm2 a--  14.00g 440.00m  3473   110          0 free

    If there is a bloc trailing after the “free” part (I had not), use:

    lvm pvmove –alloc anywhere /dev/md2:xxx-xxx

     

    Resize the device a bit larger than the logical volume. If you don’t use a sufficient size, you will get an error message:

    pvresize –setphysicalvolumesize 2.7G /dev/md2

    /dev/md2: cannot resize to 691 extents as 695 are allocated.
    0 physical volume(s) resized / 1 physical volume(s) not resized

    pvresize –setphysicalvolumesize 2.8G /dev/md2

    Physical volume "/dev/md2" changed
    1 physical volume(s) resized / 0 physical volume(s) not resized

     

    Resize the array to use less disks:

    Reduce the array to use 3 disks (-n3)

    mdadm –grow -n3 /dev/md2

    mdadm: this change will reduce the size of the array.
          use --grow --array-size first to truncate array.
          e.g. mdadm --grow /dev/md2 --array-size 7114624

    You have first to reduce the array size.

    mdadm –grow /dev/md2 –array-size 7114624

    mdadm –grow -n3 /dev/md2 –backup-file /root/mdam.md2.backup &

     

    Monitor the progress of the resizing with:

    cat /proc/mdstat

    Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [raidF1]md2 : active raid5 sdak5[4] sdaj5[3] sdai5[2] sdah5[1] sdag5[0]7114624 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU][======>..............] reshape = 31.7% (1129600/3557312) finish=0.3min speed=112960K/sec

    md1 : active raid1 sdak2[4] sdaj2[3] sdai2[2] sdah2[1] sdag2[0]2097088 blocks [12/5] [UUUUU_______]
    md0 : active raid1 sdak1[4] sdaj1[3] sdag1[0] sdah1[1] sdai1[2]2490176 blocks [12/5] [UUUUU_______]
    unused devices: <none>

    This can take a lot of time, but you can continue (I did wait 😉 )

    Finalize

    pvresize /dev/md2

    Physical volume "/dev/md2" changed
    1 physical volume(s) resized / 0 physical volume(s) not resized

    lvextend -l 100%FREE /dev/vg1000/lv

    Size of logical volume vg1/volume_1 changed from 2.70 GiB (692 extents) to 4.07 GiB (1041 extents).
    Logical volume volume_1 successfully resized.

    for btrfs:

    btrfs filesystem resize max /volume1

    for ext2/ext4:

    e2fsck -f /dev/vg1000/lv

    e2fsck 1.42.6 (21-Sep-2012)
    Pass 1: Checking inodes, blocks, and sizes

    Running additional passes to resolve blocks claimed by more than one inode...
    Pass 1B: Rescanning for multiply-claimed blocks
    Multiply-claimed block(s) in inode 13: 9221
    Pass 1C: Scanning directories for inodes with multiply-claimed blocks
    Pass 1D: Reconciling multiply-claimed blocks
    (There are 1 inodes containing multiply-claimed blocks.)

    File /@tmp (inode #13, mod time Sat May 23 23:00:14 2020)
    has 1 multiply-claimed block(s), shared with 0 file(s):
    Multiply-claimed blocks already reassigned or cloned.

    Pass 2: Checking directory structure
    Pass 3: Checking directory connectivity
    Pass 4: Checking reference counts
    Pass 5: Checking group summary information
    1.42.6-23739: 30742/179520 files (0.7% non-contiguous), 685397/720896 blocks

    Hope that no nodes are corrupted. Otherwise… well… accept to fix them.

    resize2fs /dev/vg1000/lv

    resize2fs 1.42.6 (21-Sep-2012)
    Resizing the filesystem on /dev/vg1/volume_1 to 1065984 (4k) blocks.
    The filesystem on /dev/vg1/volume_1 is now 1065984 blocks long.

    mount /dev/vg1000/lv /volume1

    Restart services:

    /usr/syno/etc.defaults/rc.sysv/S80samba.sh start

    /usr/syno/etc.defaults/rc.sysv/S83nfsd.sh start

    /usr/syno/etc.defaults/rc.sysv/pgsql.sh start

    /usr/syno/etc.defaults/rc.sysv/synomount.sh start

     

    synopkg onoffall start

     

    Remove the new  spare disks from the volume

    mdadm –detail –scan

    ARRAY /dev/md0 metadata=0.90 UUID=3b122d95:7efea8ff:3017a5a8:c86610be

    ARRAY /dev/md1 metadata=0.90 UUID=bd288153:d00708bf:3017a5a8:c86610be

    ARRAY /dev/md2 metadata=1.2 spares=2 name=DS3617_62:2 UUID=875ad2d6:956306b7:8c7ba96b:4287f6e6


    mdadm –detail /dev/md2

    /dev/md2:
           Version : 1.2
    Creation Time : Sat May 23 14:00:02 2020
        Raid Level : raid5
        Array Size : 7114624 (6.79 GiB 7.29 GB)
    Used Dev Size : 3557312 (3.39 GiB 3.64 GB)
      Raid Devices : 3
    Total Devices : 5
       Persistence : Superblock is persistent

        Update Time : Sat May 23 18:54:30 2020
             State : clean
    Active Devices : 3
    Working Devices : 5
    Failed Devices : 0
    Spare Devices : 2

             Layout : left-symmetric
        Chunk Size : 64K

               Name : DS3617_62:2  (local to host DS3617_62)
              UUID : 875ad2d6:956306b7:8c7ba96b:4287f6e6
            Events : 56

        Number   Major   Minor   RaidDevice State
          0       8        5        0      active sync   /dev/sda5
          1       8       37        1      active sync   /dev/sdc5
          2       8       53        2      active sync   /dev/sdd5
          3       8       69        -      spare   /dev/sde5
          4       8       85        -      spare   /dev/sdf5

    Here above, we see that sde5 and sdf5 are unused

     

    mdadm –fail /dev/md2 /dev/sde5

    mdadm: set /dev/sde5 faulty in /dev/md2

    mdadm –fail /dev/md2 /dev/sdf5

    mdadm: set /dev/sdf5 faulty in /dev/md2

    mdadm –remove /dev/md2 /dev/sde5

    mdadm: hot removed /dev/sde5 from /dev/md2

    mdadm –remove /dev/md2 /dev/sdf5

    mdadm: hot removed /dev/sdf5 from /dev/md2

     

    Et voilà.

    Loading

  • Add Support for SHR on Synology DS3617xs

    I wanted to test a procedure to shrink a SHR volume on a virtual Synology DS3617. Unfortunately, SHR is not an available option on the high end models of Synology. But it can be enabled easily 😉

    Click to Read More

    DS3617xs is a model where SHR is not enabled:

    0KenZdg.png

    To enable it:

    1. Open a SSH console and enter root mode.
    2. Edit the file  /etc.defaults/synoinfo.conf
    3. At the end, you should find: supportraidgroup=”yes”
    4. Comment that line with a #
    5. Next to it, add a new line with: support_syno_hybrid_raid=”yes”
    6. Reboot your Synology

    You can use “vi” to edit the file: sudo vi /etc.defaults/synoinfo.conf,

    • Press i or the insert key to make your modifications, then “Esc” to end the modifications.
    • After that type :x! to save and exit. Ot :q! to exit without saving.

    Et voilà, your DS3617xs offers now the option for SHR

    Loading

  • VMWare Workstation not compatible with Windows 10

    Starting a VM this morning, I got a new message “VMware Workstation and Device/Credential Guard are not compatible”. To be able to run my VM, I had to disable this Credential Guard.

    Click to Read More

    Credential Guard uses virtualization to store credentials in protected containers separate from the OS. As a result, the information Credential Guard protects is safe even if malware or some other malicious attack penetrates an organization’s network.

    But it prevents VMWare to run VM’s.

    To disable it, copy this script in a file named “disableCG.ps1” in your personal home folder C:\Users\<Your Name>\

    Run next a PowerShell as Administrator:

    Pay Attention that this will reboot automatically your PC !!

    In this PowerShell, execute: cd ~

    Next execute : .\disableCG.ps1 -Disable -AutoReboot

    You will see errors displayed in Red in the PowerShell. This won’t be a problem. And after a minute, the PC will reboot.

    During the reboot, you will be prompted to confirm that you really want to disable the Windows Defender Credential Guard. Press the Windows Key or F3 to confirm.

     

    Other options : https://windowsreport.com/disable-credential-guard-windows-10/

    Ex. Execute in a PowerShell run as Administrator: bcdedit /set hypervisorlaunchtype off

     

    I also had to remove “Guarded Host” and “Hyper -V” via the Control Panel > Programs > Turn Windows features on or off

    But now, Docker does not run anymore on my PC as it needs Hyper-V 🙁

    Loading

  • Best Package delivery price

    Looking for the best price to deliver a package ?

    Packlink.com allows to compare the costs for shipping parcels from everywhere to everywhere: https://www.packlink.com/

    Loading

  • Philips Hue lights Bulbs and Accessories become unreachable

    Regularly, my Philips Hue lights and accessories do not respond anymore. Looking into the Philips Hue App, they appear as “Unreachable” (no link). The solution was to avoid WiFi interference!

    Click to Read More

    In French, the message displayed by the App, for the Lights and accessories, is “impossible d’établir le lien”.

    If your Philips Hue lights aren't responding, maybe it's time to ...

     

    Notice that each time, a few bulbs and accessories were still connected and the App was perfectly able to connect onto the Bridge, both via Internet and via my local Network. Notice also that I have many Philips Hue Light Bulbs, Sensors and Dimmer Switches in the house (less than 50, which is the theoretical maximum for a Philips Hue Bridge) as well as a few Tradfri Bulbs in the corridors.

    I really spent hours to investigate this issue which occurred once every few months …

    • Unplug the power supply of the Philips Bridge for 30 sec.
    • Switch off all the lights for 30 sec.
    • Remove the Bulbs from their sockets.
    • Press the 4 buttons of the remotes (Philips Hue Dimmer Switch) all together for a few seconds (until the led start to blink red/green).
    • Press both the On and Off buttons of the remote to reset the Bulbs.
    • Turn on/off the lights quickly several times (and do an incantation for Apollo).
    • Delete and Re-add again the lights and accessories within the App.
    • Go through the whole FAQ of Philips Hue.
    • Drink a bottle of wine and cry alone in my sofa.
    • etc…

    I was always finally switching off all electric devices which were possibly disrupting the Zigbee signals : my routers, my wifi access points, my other Bridges (Tahoma Somfy, Z-Wave, …) … and even powering off the whole house… This extreme action was solving the problem ! But impossible to conclude what was the precise root cause.

    Today, the problem occurred again. After various unsuccessful attempts (those mentioned above),  I read on a blog that the quality of the ethernet connection between the Philips Hue Bridge and Internet matters ?!? And I realized that my Bridge was plugged into a Devolo Powerline Adapter (Ethernet via Power Sockets). I also remember that in the past, unplugging and replugging that Devolo Powerline Adapter was an action I did and which solved the problem (but I actually did this to restart the various Bridges and WiFi access points connected onto it).

    So, I moved the Philips Hue devices from that Adapter into the main Ethernet Router and boom, it worked !

    Et voilà ? Really not sure. The ethernet connection is only for the Bridge to have access on a Wifi Access Point and be able to connect with the mobile App… The bulbs and the Bridge are connected via Zigbee. So, there  must be another reason hidden behind this (fake) “solution”.

    Note that the Phillips Hue Light Bulbs, even though they are Zigbee, do not act as repeaters for anything except other Light Bulbs on the same Hue bridge if they are connected to the Hue bridge (They use actually a “Zigbee Light Link” protocol instead of the regular Zigbee).

    As I noticed that the disconnected Bulbs are often the Tradfri ones and the Philips Hue far from the Bridge, I thought that the problem was possibly more with the Tradfri light bulbs not receiving/repeating properly the signal for the Philips Hue light bulbs ? Two possible options:

    • Replace the cheap Tradfri light bulbs with expensive Philips Hue light bulbs. I will do so soon or later…
    • Investigate why receiving/repeating the ZigBee signal could fail… I can start with this.

    It’s often mentioned on the web that ZigBee can suffer for WiFi interferences. It shares the same 2.4GHz wireless band as WiFi routers and overlaps by default with the Wifi channel 11 (https://statusq.org/archives/2018/01/09/8435/). And indeed, moving the Philips Hue Bridge or Turning On/Off all Wifi Devices in the house used to solve, at least temporarily, the problem.

    Ex.: Zigbee channels 21 to 24 overlaps with WiFi channel 11 as illustrated bellow. 

    A first obvious solution  is to disable the WiFi 2.4GHz band and use only the 5GHz band for the WiFi Devices at home. But not all of them support 5GHz…

    Another solution is to change the channel used by the Philips Hue Bridge  to use a ZigBee channel having less conflict with your WiFi (All lights must be powered on). Do the change via Philips Hue App’s settings > Hue Bridges > touch the “i” of the Bridge to update > ZibBee channel change > Change Channel.

    It will take about 30 seconds, but the Hue system will change the channel to another one automatically. Once done, the lights should all come back online (And indeed, this did the trick once for me too, without doing any other action). If they don’t, turn them on and off again at the main light switch/plug socket. Possible Zigbee channels are 11, 15, 20 and 25

    Philips Hue Zigbee channel

    Also you can try to move the Philips Hue Bridge to another location, not too close of the WiFi access points and in central location regarding the various ZigBee devices. This is what I did by connecting the Bridge directly on the Router (So the solution was not the improvement of the ethernet connection quality, but most probably a lower WiFi interference next to that Router).

    Notice that this won’t update the channel used by the Tradfri lights! As explained on Reddit by Erik1971, you can try, after changing the Hue System channel, to power cycle the Tradfri lights (make sure they are not powered for at least 1 minute). This should trigger the Tradfri lights to scan over the channels to join the network again. Best way to do this is to power off all Tradfri lights and keep the Hue Lights and Bridge powered and then one by one power on the Tradfri lights (to prevent that the Tradfri Lights see the other Tradfri lights on the wrong channel and rejoin on the wrong channel).

    So, for the time being, my conclusion is that bulbs and accessories became unreachable due to (many?) new devices connecting on the WiFi or WiFi Access Points changing automatically their channels (This can be checked with an App like Wifi Analyzer).

    Loading

  • Backup Synology to Unraid

    The easiest way to backup a Synology NAS to Unraid Server is to use Hyper Backup on Synology and rsync on Unraid.

    Click to Read More

    First, enable rsync on your Unraid Server. It is preinstalled but not running as a daemon.

    Create a file /boot/custom/etc/rsyncd.conf with the following content:

    uid             = root
    gid             = root
    use chroot      = no
    max connections = 4
    pid file        = /var/run/rsyncd.pid
    timeout         = 600
    
    [backups]
        path = /mnt/user/backups
        comment = Backups
        read only = FALSE

    Here above:

    • The name “backups” between brackets will be visible as “backup module” from the Synology. You can create several blocks like this one.
    • The “path” (here /mnt/user/backups) must exist on your Unraid server (create this one as a shared folder, to be able to access the backup later from any PC)
    • Notice: the folder /boot should exist. But you could possibly have to create the subfolders /custom/etc

     

    Next, create a file /boot/custom/etc/rc.d/S20-init.rsyncd with the following content:

    #!/bin/bash
    
    if ! grep ^rsync /etc/inetd.conf > /dev/null ; then
    cat <<-EOF >> /etc/inetd.conf
    rsync   stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/bin/rsync --daemon
    EOF
    read PID < /var/run/inetd.pid
    kill -1 ${PID}
    fi
    
    cp /boot/custom/etc/rsyncd.conf /etc/rsyncd.conf

    Finally, add the following line in the file /boot/config/go :

    #!/bin/bash
    bash /boot/custom/etc/rc.d/S20-init.rsyncd

     

    Now, either reboot or execute: bash /boot/custom/etc/rc.d/S20-init.rsyncd

     

    Go now on your Synology and open “Hyper Backup” to Create a new Data Backup Task:

    Select rsync as Backup Destination:

    And Create the backup Task with “rsync-compatible server” as Server Type:

    In order to access the backup and retrieve files from a PC, use the application “Hyper Backup Explorer” from Synology and open the backup file .bkpi located under \\<YourServer>\backups\<Yourbackup>.bkp\

    Loading