Synology Synology: how to get MAC address from IP address and vice versa

Here is how to retrieve the IP address of devices in the same LAN as your Synology, based on their a MAC addresses, and vice-versa.

Click to Read More

I am using a php script to executes various bash scripts. it requires the package inetutils that can be installed using ipkg. Type in a ssh console run as root: /opt/bin/ipkg install inetutils

<?php
//Clean all MAC addresses from ARP table
function FlushArpTable()
{
    //only work if run as root
    $flush = exec("ip -s -s neigh flush all");
}

//Return 1 if the given $ip is a local one
function IsLocalIp($ip)
{
    //Get all local IP address from ifconfig
    //$localIp = `ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '127.0.0.1'`;
    $localIp = exec("ifconfig | grep -o $ip");
    return ($localIp == $ip);
}

//Ping $ip using inetutils (only this one works when not run as root)
//return -1 if package inetutils is not installed (can be done using ipkg)
function Ping($ip)
{
    $ping = exec("if [ -f /opt/bin/ping ] ; then /opt/bin/ping -c1 $ip; echo 1; else echo -1; fi");
    return $ping;
}

//Return MAC address related to the given $ip address in ARP table or ifconfig
//Ping the $ip if MAC is not found and $force=1
//NB.: Synology local ip are not added in the ARP table, hence the use of ifconfig
function GetMacFromArp($ip, $force = 1)
{
    if (filter_var($ip, FILTER_VALIDATE_IP)) {
        // Look into the arp table for the MAC address linked to the given ip address
        $mac = exec("arp -a $ip | grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}'");
        $mac = trim($mac);
        
        if ($mac == "") {
            //if the ip is not in the ARP table, look for it via ifconfig
            $mac = `ifconfig | grep -E "HWaddr |inet |^$" | awk '$1=$1' ORS="\n\n" RS="\n\n" | grep $ip | grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}'`;
            $mac = trim($mac);
        }
        
        if ($mac == "" && $force == 1) {
            echo "? ";
            Ping($ip);
            $mac = GetMacFromArp($ip, $force = 0);
        } else if ($force == 1) {
            echo "! ";
        }
    }
    
    return $mac;
}

//Return ip address related to the given $mac address in ARP table or ifconfig
//NB.: Synology local ip are not added in the ARP table, hence the use of ifconfig
function GetIpFromArp($mac)
{
    if (filter_var($mac, FILTER_VALIDATE_MAC)) {
        // Look into the arp table for the ip linked to the given mac address
        $ip = exec("arp -a | grep $mac | grep -o -E '\b([0-9]{1,3}\.){3}[0-9]{1,3}\b'");
        
        if ($ip == "") {
            // if no ip found in the arp table, look for it into the ifconfig
            $ip = exec("ifconfig | grep -E 'HWaddr |inet |^$' | awk '$1=$1' ORS='\n\n' RS='\n\n' | grep $mac | grep -o -E 'addr:\b([0-9]{1,3}\.){3}[0-9]{1,3}\b' | grep -o -E '\b([0-9]{1,3}\.){3}[0-9]{1,3}\b'");
        }
    }
    
    return $ip;
}

function DisplayMac($ip)
{
    if (filter_var($ip, FILTER_VALIDATE_IP)) {
        $mac = GetMacFromArp($ip);
        if (filter_var($mac, FILTER_VALIDATE_MAC)) {
            echo "MAC of $ip is " . $mac;
            if ($ip != GetIpFromArp($mac)) {
                echo " (? " . GetIpFromArp($mac) . " ?)";
            }
        } else {
            echo "MAC of $ip is unknown";
        }
        if (IsLocalIp($ip)) {
            echo " (a local IP)";
        }
        echo "<br/>";
    }
}

// Check if  ping works
if (Ping("127.0.0.1") == -1) {
    echo "You have to install inetutils. Ex.: using ipkg";
} else {
    //clean the arp table (only work when run as root)
    FlushArpTable();
    
    //Display the mac address of an ip in the LAN
    DisplayMac('192.168.0.19');
    
    //get ip of all local interfaces
    $localIps = split("\n", `ifconfig | awk '/inet addr/{print substr($2,6)}'`);
    foreach ($localIps as $ip) {
        DisplayMac($ip);
    }
    
    DisplayMac('192.168.0.47');
    
}
?>

Synology Synology: System internal service [apparmor] failed to start

Since I have upgraded my DSM 6.1.4 to the latest update, I have this error : "System internal service [apparmor] failed to start."

I did contact Synology ans the only solution is to reintall the DSM

Click to Read More

To confirm the problem, open a SSH console and enter the root mode as explained here.

In that console, type: synoservice --is-all-up

You should see that the apparmor service didn't start properly after the reboot:

root@YourNas:~# synoservice --is-all-up
service [ "apparmor" ] failed on bootup.

 

Here is the answer from Synology

Good evening,

 

thanks for contacting the Synology technical support and sorry for the late reply, we're having an high volume of ticket.

We're aware of this issue, did you recently update your DSM?

Anyway, I'm sorry but you will have to reinstall DSM to fix this issue, please take a full backup of your Data using "Hyper backup" or "Cloud Sync".

After that navigate to Control Panel -> Update and Restore -> Configuration Backup -> Backup Configuration-> Reset -> Erase all Data

Reinstall DSM and restore Configuration and Data.
Take in mind that the response time might be delayed a bit because of the high amount of Ticket that we received.

Thanks for your understanding and for your patience.
If you have any other question or doubt, don't hesitate to contact me back.

Best regards,
xxxxxxxx

Technical support FR

Synology Unattended GPG key generation to sign Synology Packages

I am going to add a feature in my own version of SSPKS to sign Synology packages. But to do this, I need a way to create GPG key from a non-interactive script.

Click to Read More

I found the required documentation here and a sample here. I did adapt that sample to:

  • not require a passphrase as mentioned in the Synology documentation. This would break the build process.
  • to generate some entropy without the rng-tools (not available out-of-the-box for Synology)

Obviously, you need gpg. I am using gpg2 made available by installing the package gpgme via ipkg as explained here.

Create keys in your personal key rings

First, put the parameters to be used by gpg into a file named 'gpgKey' (don't forget that it must be linux compliant => not CRLF (/r/n) but only LF (/n)*). Here under, I am using RSA 2048. Use your own name, comment and email address.

Key-Type: RSA
Key-Length: 2048
Subkey-Type: RSA
Subkey-Length: 2048
Name-Real: Type Here your name
Name-Comment: Type Here some comment
Name-Email: Type Here your email
Expire-Date: 0
%no-protection
%commit

Next, save the following script in a file named 'createGpgKey.sh' (don't forget about CRLF ;) )

#!/usr/bin/env bash
if [ ! -f ~/.gnupg/gpg.conf ]; then
mkdir -p -m 0700 ~/.gnupg
touch ~/.gnupg/gpg.conf
chmod 600 ~/.gnupg/gpg.conf
tail -n +4 /opt/share/gnupg/gpg-conf.skel > ~/.gnupg/gpg.conf

touch ~/.gnupg/{pub,sec}ring.gpg
fi

#generate some entropy
(dd if=/dev/zero of=/dev/null) & pid=$!

#generate the key (and a folder to be used later ;)
mkdir -p -m 0700 gpg
gpg2 --verbose --batch --gen-key ./gpgkey

#kill the entropy generator
kill $pid

Then, move those two files into a public shared folder of your Synology. Ex.: '\\<Your Nas>\temp'

And open a ssh console as explained here (no need to enter the root mode) to go into the shared folder: cd /var/services/temp

Finally, here is the output you should see when running your script: sh createGpgKey.sh

gpg: skipping control `%no-protection' ()
gpg: writing self signature
gpg: RSA/SHA1 signature from: "1418FFE0 [?]"
gpg: writing key binding signature
gpg: RSA/SHA1 signature from: "1418FFE0 [?]"
gpg: RSA/SHA1 signature from: "71C22B00 [?]"
gpg: writing key binding signature
gpg: RSA/SHA1 signature from: "1418FFE0 [?]"
gpg: RSA/SHA1 signature from: "71C22B00 [?]"
gpg: writing public key to `/var/services/homes/<current user>/.gnupg/pubring.gpg'
gpg: writing secret key to `/var/services/homes/<current user>/.gnupg/secring.gpg'
gpg: /var/services/homes/<current user>/.gnupg/trustdb.gpg: trustdb created
gpg: using PGP trust model
gpg: key 1418FFE0 marked as ultimately trusted

The keys are now stored into your home's gpg folder: ls ~/.gnupg/

You can check that the key id displayed above are stored using:

  • gpg2 --list-keys

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
/var/services/homes/<current user>/.gnupg/pubring.gpg
--------------------------------------------
pub 2048R/1418FFE0 2017-12-26
uid Your Name (Your Comment) Your Email Address
sub 2048R/71C22B00 2017-12-26

  • gpg2 --list-secret-keys

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 2 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 2u
/var/services/homes/<current user>/.gnupg/secring.gpg
--------------------------------------------
sec 2048R/1418FFE0 2017-12-26
uid Your Name (Your Comment) Your Email Address
ssb 2048R/71C22B00 2017-12-26

*: the trick to replace all CRLF by LF is to edit the file with notepad++, use the menu Encoding > Convert to UTF-8, and next do CTRL-H to search and replace /r/n with /n using the search mode = Extended. You can finally check that there is no more CRLF by using the menu View > Show Symbol > Show End of Line.

NB.: the script could be a bit slow if there is not enough entropy (up to 45sec on my NAS)... (to download a big file within the download station can help ;)

You can now export your public key using either your name, your email or the key id (from illustration here above, the key id is 1418FFE0): gpg2 --armor --export 1418FFE0 > gpgkey.asc

This file 'gpgkey.asc' must be copied in the root folder of your SSPKS server, so it will automatically be added as a trusted publisher for anyone who is adding your SSPKS url in his list of "Package Sources".

Now, assume that you have the package to be signed (a spk file) and the 'CodeSign.php' script from Synology's toolkit (it can be downloaded from github) in the same shared folder 'temp' used previously... Then, you can sign this package using: php CodeSign.php --sign=YourPackage.spk --keydir=~/.gnupg --keyfpr=1418FFE0

Create keys in specific key rings

Instead of using your personal key rings, you could prefer to store your keys in a dedicated folder... To do so, you can use the following parameters in your gpgKey file:

Key-Type: RSA
Key-Length: 2048
Subkey-Type: RSA
Subkey-Length: 2048
Name-Real: Type Here your name
Name-Comment: Type Here some comment
Name-Email: Type Here your email
Expire-Date: 0
%no-protection
# Write public key into a file pub.gpg
%pubring gpg/pubring.gpg
# Write secret key into a file sec.gpg
%secring gpg/secring.gpg
%commit

Once the script createGpgKey.sh executed, you will find two files pubring.gpg and secring.gpg in the folder gpg (created by the script that run in /var/services/temp/).

gpg: Generating a basic OpenPGP key
gpg: skipping control `%no-protection' ()
gpg: writing public key to `gpg/pubring.gpg'
gpg: writing secret key to `gpg/secring.gpg'
gpg: writing self signature
gpg: RSA/SHA1 signature from: "F93A0789 [?]"
gpg: writing key binding signature
gpg: RSA/SHA1 signature from: "F93A0789 [?]"
gpg: RSA/SHA1 signature from: "8AAC41CF [?]"
gpg: writing key binding signature
gpg: RSA/SHA1 signature from: "F93A0789 [?]"
gpg: RSA/SHA1 signature from: "8AAC41CF [?]"

You can now sign your package using: php CodeSign.php --sign=YourPackage.spk --keydir=/var/services/temp/gpg --keyfpr=F93A0789

You can also check your keys using:

  • gpg2 --no-default-keyring --secret-keyring ./gpg/secring.gpg --keyring ./gpg/pubring.gpg --list-secret-keys
  • gpg2 --no-default-keyring --secret-keyring ./gpg/secring.gpg --keyring ./gpg/pubring.gpg --list-keys

And you can export your public key to be used in your SSPKS using:

  • gpg2 --no-default-keyring --keyring ./gpg/pubring.gpg --armor --export F93A0789 > gpgkey.asc

Create keys with a passphrase

In order to provide a passphrase, replace '%no-protection' in the parameters file with:

Passphrase: Type Here your passphrase

Synology Install the Synology Toolkit and Sign SPK packages

Since a long time, I was waiting to have enough free time to investigate how to sign my own packages with Synology's toolkit. It's now finally done!

Click to Read More

Initial installations

  1. First, install IPKG as explained here.
  2. Next, install the official Synology package "python3"
    1. Alternative: install python3 using the command: ipkg install python3
  3. Then, create a symlink into /usr: ln -s /usr/local/bin/python3 /usr/bin/python3
    1. if using python installed with ipkg, create a symlink into /usr: ln -s /opt/bin/python3 /usr/bin/python3
  4. install gpg using the command: ipkg install gpgme
    1. This was needed as gpg was installed on my NAS without gpg-agent (? to be investigated later)...
  5. Now, install GIT as explained here.
  6. Finally run:
    1. cd /volume1/
      1. There was not enough free space in the system partition to install the toolkit!!
    2. mkdir -p toolkit
    3. cd toolkit/
    4. git clone https://github.com/SynologyOpenSource/pkgscripts-ng pkgscripts
    5. cd pkgscripts/

If you want to use the toolkit for other purpose than signing

You can check the available platform for your DSM version in the toolkit (e.g. here for 6.1). Run : ./EnvDeploy -v 6.1 --list

It should display a list like this one:

Available platforms: 6281 alpine alpine4k armada370 armada375 armada38x armadaxp avoton braswell broadwell bromolow cedarview comcerto2k dockerx64 evansport grantley hi3535 kvmx64 monaco qoriq x64 rtd1296 denverton apollolake

  • N.B.: The major and minor version (E.g. 6.1 for major=6 and minor =1) of your DSM can be found using: cat /etc.defaults/VERSION
  • N.B.: the platform of your Synology can be found in this list based on your model (See the column "Package Arch"). The model of your Synology can be found running the command:  cat /etc.defaults/synoinfo.conf | grep -m 1 'upnpmodelname' | cut -d "=" -f 2
  • N.B.: You can also check the CPU of your Synology using: cat /proc/cpuinfo | grep -m 1 'model name' | cut -d ":" -f 2 | cut -d "@" -f 1

And you can configure the toolkit by running the following command with your DSM version (E.g.: 6.1) and platform (E.g.: avoton) as parameter: ./EnvDeploy -v 6.1 -p avoton

  • NB: the platform must be in lower case !!!

In order to sign packages

You have to create a GPG key. Type: /opt/bin/gpg2 --gen-key

Once prompted, choose:

  • RSA and RSA (default)
  • key size 1024, 2048 or 4096
  • key does not expire
  • And do not type any passphrase. Just press 'Enter' without typing any character until it accepts to not make your key secure.

gpg (GnuPG) 2.0.18; Copyright (C) 2011 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 2048
Requested keysize is 2048 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: vletroye
Email address: <obfuscated>
Comment: BeatificaBytes' Packages
You selected this USER-ID:
"vletroye <obfuscated>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.

You don't want a passphrase - this is probably a *bad* idea!
I will do it anyway. You can change your passphrase at any time,
using this program with the option "--edit-key".

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: key 9ABC9981 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
pub 2048R/9C293482 2017-12-23
Key fingerprint = 4AB4 051C EF8D 009D 4BF3 D78F EEBC 5B44 9C29 3481
uid vletroye (BeatificaBytes' Packages) <obfuscated>
sub 2048R/4C18D693 2017-12-23

You can now:

  • view public keys using: gpg2 --list-keys
  • view private keys using: gpg2 --list-secret-keys
  • delete a private key: gpg2 --delete-secret-key <user name> (you can also use the <email> or <key id>, displayed after pub xxxxR/ where xxxx is the keysize, e.g.: 1024)
    • Ex.: gpg2 --delete-secret-key vletroye
    • Ex.: gpg2 --delete-secret-key 9C293482
  • delete a public key: gpg2 --delete-key <user name>
  • You can also use specific rings (Ex.: if you did copy them somewhere else than the default user's home folder : ~/.gnupg)
    • gpg2 --no-default-keyring --secret-keyring /root/.gnupg/secring.gpg --list-keys
    • gpg2 --no-default-keyring --secret-keyring /root/.gnupg/secring.gpg --list-secret-keys
    • gpg2 --no-default-keyring --keyring /root/.gnupg/secring.gpg --list-keys
    • gpg2 --no-default-keyring --keyring /root/.gnupg/secring.gpg --list-secret-keys

And last but not least... you can finally sign a package located e.g. under /web/packages, using: php CodeSign.php --sign=/volume1/web/packages/YourPackage.spk --keydir=/root/.gnupg --keyfpr=9C293482

Now that this package is signed, add your certificate into the list of trusted publishers within your Synology's Package Center.

  1. First, run: gpg2 --armor --export <user name> > YourCertificate.asc
  2. Next copy this one into a public shared folder. E.g.: cp YourCertificate.asc /volume1/web/packages/
  3. Finally:
    1. import it: Package Center > Settings > Certificate > Import > Browse >
    2. and only trust Synology Inc. and trusted publishers: Package Center > Settings > General
Import Certificate

Import Certificate

Trusted Publishers

Trusted Publishers

 

Instead of manually importing your certificate as illustrated above, if you are using SSPKS to distribute your packages, you can copy it there: cp YourCertificate.asc /var/services/web/sspks/gpgkey.asc

Doing so, it will automatically be added as a trusted publisher for anyone who is adding your SSPKS url in his list of "Package Sources".

 

Synology Plex Server not found after Synology update to DSM 1.10.1.4602

After updating DSM on my Synology, Plex Media Player was not able to find the Plex Server running there.

Plex Server Not Found

Click to Read More

Plex Server was clearly running as I was able to connect via http://<My Synology>:32400/web/index.html

But it was not responding to any request to display Libraries or to play any Movies... To solve this, I had to :

  1. Stop the package Plex via the DSM Package Center
  2. Log in my NAS as an administrator, in a SSH console, via Putty and enter root mode.
  3. Find Plex process still running via the command: ps -gaceux | grep plex
    1. This command displayed: plex 13150 0.2 0.2 677640 17832 ? Sl Dec17 7:41 Plex Media Server
  4. Kill plex's remaining process: kill -9 13150
  5. Possibly do also delete the Plex's Plugin Service as done here.
  6. Start the package Plex via the DSM Package Center

Et voilà.

IMO, the problem was that I didn't stop properly Plex Package before updating my DSM. The Plex instance running at that time apparently didn't completely shutdown. But it was not really running fine anymore either. Unfortunately, that instance was preventing a new one to start (as it was still using the port 32400).

Synology Plex Media Server not matching/downloading metadata anymore

Since a few days, My Plex Media Server, hosted on my Synology, was not able to match movies anymore. And when matching movies manually, it was sometimes downloading the covers but never the metadata. Deleting the Service plugin solved the problem.

Click to Read More

I read that IPV6 could be the cause of such a problem, and that turning it off on the Synloloy was the solution. I tried that but without success.

Was solved my problem was to:

  1. Stop Plex via the DSM's Package Manager
  2. Log in my NAS as an administrator, in a SSH console, via Putty and enter root mode.
  3. Enter Plex's Plugin folder using a command like: cd /volume1/Plex/Library/Application\ Support/Plex\ Media\ Server/Plug-ins/
  4. Delete the service plugin:  rm -R Services.bundle
  5. Restart Plex via the DSM's Package Manager

Synology How to capture a Synology Package during installation

I recently discovered GateOne, a free HTML5 web-based terminal emulator and SSH client for which there is a Package for Synology on the SynoCommunity. Unfortunately, the setup doesn't work anymore on the most recent DSM versions. As SynoCommunity prevents users to manually download their SPK, I did recover it from Synology's temp download in order to fix myself the issues reported here and recreate a package.

[EDIT 30/12/2018] I have posted a new version of this script here.

Click to Read More

So, as one cannot manually download the Synology package (SPK file), the idea is to simply capture the SPK during its installation (even if that installation fails).

First, log into a SSH console via putty and enter root mode.

Create the following script, named 'capture.sh'

#!/bin/bash
VOLUME=$1
TARGET=$2

if [ -z "$VOLUME" ]; then
echo "usage: capture <source VOLUME> <target dir>"
exit 0
fi

if [ -z "$TARGET" ]; then
echo "usage: capture <source VOLUME> <target dir>"
exit 0
fi

pattern="volume[0-9]+"
if [[ ! $VOLUME =~ $pattern ]]; then
echo "The name of the source volume must be like 'volume<i>' where <i> is numeric"
exit 0
fi

if [ ! -d "/$VOLUME/@tmp" ]; then
echo "Temporary dir not found: /$VOLUME/@tmp"
exit 0
fi

if find "/$VOLUME/@tmp/" -mindepth 1 -print -quit | grep -q .; then
rm -R /$VOLUME/@tmp/*
fi

echo "Press any key to stop the capture"

if [ -t 0 ]; then stty -echo -icanon -icrnl time 0 min 0; fi

count=0
keypress=''
while [ "x$keypress" = "x" ]; do
let count+=1
echo -ne $count'\r'
keypress="`cat -v`"
cp -R "/$VOLUME/@tmp" "$TARGET/SynoCapture"
done

if [ -t 0 ]; then stty sane; fi

echo "$count captures done"
exit 0

This script will

  1. first, delete the temporary folder from the volume where you will install the package to be captured. Ex.: /volume1/@tmp
  2. next, loop until you press any key. And during the loop, it will copy any (new) content from the temporary folder into a folder named 'SynoCapture' on a target shared folder. Ex.: on your NAS' web folder /var/services/web/

So, concretely,

  1. Run the script via a command like: ./capture volume1 /var/services/web/
  2. Install your package via the package manager
  3. Stop the script by pressing any key once the package has been installed
  4. (You may now uninstall your package)

Voilà, first step accomplished. Now, look into your capture (Ex.: in \\<Your NAS>\web\SynoCapture\).

  1. You should see a folder named "@synopkg\@download\<Package>". This folder contains the package but without its file extension .spk (Ex.: @SYNOPKG_DOWNLOAD_<Package>).  Gotcha !!!
  2. You should also see a folder named "pkginstall". This one contains the package deflated.
  3. Now, a little extra... you should also find a folder name "pkglist.tmp" containing two files: synoserver.enu and : otherserver.enu (with possibly other extensions depending on your DSM language?). Those files contains the URL's of all SPK available respectively on the official Synology website and on the various SPK servers that you configured in your Package Center > Settings > Package Sources.

If you are running Windows 10 with Powershell, copy the content of those files into on file named input.txt and open a powershell in the folder where you have saved that input.txt file. Execute next this to filter the file and extract the URL's:

select-string -Path input.txt -Pattern 'http[^"]*\.spk[^"]*' -AllMatches | % { $_.Matches } | select-object Value > output.txt

You can copy/paste the content of output.txt into MyJDownloader (asking for a "deep link analysis"!). Magic !

SynologyWordpress WordPress RewriteRule to "Redirect" after moving the blog

I did reconfigure my Synology Web Stations to access my blog directly at www.beatificabytes.be instead of www.beatificabytes.be/wordpress/. The problem was to redirect next the old pages still referenced by google onto their new address. Ex.: www.beatificabytes.be/wordpress/softwares onto www.beatificabytes.be/softwares. Using a RewriteRule did the trick.

Click to Read More

Concretely, I did configure a Virtual Host in the Web Station

WebStation VirtualHost

Next, I did reconfigure wordpress's wp-config.php by replacing:

define('WP_HOME','/wordpress');
define('WP_SITEURL','/wordpress');
define('WP_SITEURL', $pageURL . '/wordpress');

with:

define('WP_HOME','http://www.BeatificaBytes.be');
define('WP_SITEURL','http://www.BeatificaBytes.be');
define('WP_SITEURL', $pageURL');

Finally, I did remove "/wordpress" from all URL's in my own posts. I didn't use a Search & Replace plugin like the "Better Search and Replace" to do the update directly in the database. Instead, I did use the plugin "Broken Link Checker" to Bulk Edit broken URLs.

Bulk Edit Broken URLIt took more time but was a much more "clean" approach as I only touched for sure broken links. (Notice: This Plugin runs in the background so you need patience for it to get the complete list of broken URLs. What' really great is that for broken external links, it suggests the replacement by an archived version of the missing pages - via the WayBack Machine)

WayBack Machine

NB.: Here is another nice trick to list all broken URL's in a website

wget --spider -o /var/log/wget.log -e robots=off -w 1 -r -p http://<youraddress>

Finally, I had to redirect all old pages still referenced on Google to their new location. The obvious solution was to use a RewriteRule to be added in the .htaccess file used by WordPress (In my case, in /var/services/web/wordpress/.htaccess). Unfortunately, it took me hours to find the trick as, instead of reading the whole apache's RewriteEngine documentation, I tried to figure out myself how to do it, based on samples found on the web.

Finally, I read these french documentations:

And it's only by reading the documentation about the flag that I understood the need to use the special flag [R] (I already tried at least more than 20 various rules before understanding I had to specify the flag R(edirect)...)

So, here it is: RewriteRule ^wordpress/(.*)$ /$1 [R,L]

  • Match any path starting with "wordpress/" and followed by "anything else"
  • Replace it with the part matching "anything else"
  • And if a match was found, do a R(edirect) and this was the L(ast) rule to apply (i.e.: stop).

I didn't change anything else. Also, changes in the .htaccess are taken into account immediately, without the need to restart the WebStation.

# Synology PHP
AddHandler default-handler .htm .html .shtml
AddHandler php-fastcgi .php
AddType text/html .php
Action php-fastcgi /php56-fpm-handler.fcgi
# Synology PHP

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^wordpress/(.*)$ /$1 [R,L]
RewriteRule ^index\.php$ - [L] RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L] </IfModule>

# END WordPress

Et voilà.

NB.: Here is a useful website to test online .htaccess RewriteRule's

Synology Synology Package Sample to illustrate how scripts and wizards are executed

I had troubles to use variables in various scripts or to create wizards via shell scripts. So I have created a dummy package logging useful information in a file to understand what's going on during the installation, the upgrade and the uninstallation of a Synology Package.

Click to Read More

This package is named MODS SPK Tracer and is available on my SPK Server. Obviously, i has been created with my Synology Package Creator tool for Windows ;)

Once installed, upgraded or removed (and obviously reinstalled), you can access its log file either by clicking on its icon in DSM's Main Menu, or by via the Package Center.

Mods SPK Tracer

This package comes with shell scripts creating on the fly "installation", "upgrade" and "uninstallation" wizard (install_uifile.sh, uninstall_uifile.sh, upgrade_uifile.sh). The purpose is to illustrate how to inject environment variables in the JSON wizard and how to use php in such scripts. Briefly said, they must generate a JSON wizard and store it in the standard log file $SYNOPKG_TEMP_LOGFILE

While testing those wizard scripts, I have discovered that, at runtime, only the running script is stored in the temporary folder. The other ones are not available. No way therefore to call another script from the running one.

The Wizards are exporting one variable to the installation scripts, for illustration purpose (I.e.: how to prompt the user for some input to be used later by the scripts)

Finally, the wizard scripts also logs the environment variables exported for them by the DSM. Not a lot are available during the execution of the wizard because the SPK is actually not yet expanded.

The package comes next with the standard installation scripts, calling all a generic script that log the environment variables exported for them by the DSM. So, you can see in which script you have access to which variable. The scripts preinst, postinst, preuninst, postuninst, preupgrade and postupgrade have access to the same variables. But it's not the case for the start-stop-status script.

Those scripts are also displaying info during the installation/upgrade/uninstallation process for the user. For that purpose, they write in the standard log file $SYNOPKG_TEMP_LOGFILE. Obviously the wizard scripts can't use this trick to display information to the user. But they don't need that as they actually display real screens (rendition of their JSON).

Notice: often, variables accessible for a script must be persisted in a file to be retrieved later by another script. Ex.: the postinst script can store a variable received from the install_uifile wizard in a file so it can be read by the postupgrade or postuninst script later.

Notice: the log file is only available in the Package Center if its path is returned by the start-stop-status script.

Notice: the INFO file of this Package is quite standard.

What's interesting is to see the order of execution. Ex.: for an upgrade

SPK Tracer order

Also, during an upgrade, the script to stop, preuninst and postuninst are the versions of the previous package, not the versions of the new package !!! (Obviously).

During an upgrade, preupgrade and preinst scripts are executed from a temporary folder (/volume1/@tmp/pkginstall/scripts), while postupgrade, postinst, and the script to start, are executed from the target installation directory (/var/packages/<Package Name>/scripts)