Tips EdgeRouter X - Speed limited to 250Mbps instead of 1Gbps

I have upgraded my Internet abonnement from 100Mbps to 1000Mbps but couldn't get more than 250Mbps. The reason was not an issue with the fiber or the modem but a limitation on my EdgeRouter X.

Click to Read More

I bought this Ubiquiti router to have the capability to view the bandwidth on each port as well as to fine-tune the configuration if required. Its OS "EdgeMAX" is indeed very flexible and powerful. Obviously, I did check that it was capable to deliver 1Gbps. 

It was clear for me that it would be "1Gbps for all ports" (4 ports). But as the Router is connected on a Switch "1Gbps per port", that was not an issue for my LAN (And it was much cheaper).

Nevertheless, having max 250Mbps was disappointing.

 

This throughput was reached with defaults settings, as I didn't fine tune anything so far.

The solution was to disable the "hardware offload" as well as the "flow-accounting (NetFlow)" (NB.: QoS may also not be used, but it was the case).

To disable these features, click on 'CLI' in the upper-right corner to open a console and login with your administrator account. Then type:

configure
set system offload hwnat enable
delete system flow-accounting
commit
save
exit
reboot

Once the router as completed the reboot, you get the maximum bandwidth !

Et voilà.

Tips PC Windows 10 is very slow although CPU, RAM, GPU and Disk usage is low

There are many many reasons why a PC can be lagging and very slow. But the worst was is when you look at the Task Manager and you see that both CPU and GPU are low, there are still plenty of free RAM and there is no high disk activity. In my case the problem was due to CPU Frequency being reduced.

Click to Read More

First thing to check, the CPU/GPU/Ram and Disk Usage. Press CTRL-SHIFT-ESC to open the Task Manager.

You see that everything is low. So what ?

Run now the Resource Monitor by pressing Windows-R and typing: resmon

Very interesting. We see that the CPU runs at 19% of its maximum Frequency. So, something has reduced the CPU frequency. Indeed, with Intel speedstep, power saving and everything else disabled, this should always read 100%.

So, the problem can be the "power plan" or some "failed" overclocking...

You can test this by adding some load on the CPU with "CPU Grab" or "CPU Stres". Here under, using CPU Grab to load the CPU at 100%, we can see that the CPU is still limited to ~20%:

NB.: You can also give a try to Intel® Extreme Tuning Utility

Regarding SpeedStep and Power Saving, you can edit your Power Plan Option. Press Windows-R and enter: powercfg.cpl, then click "Change plan settings" > "Change advanced power settings". Set all limits to 100%.

NB.: For laptop, there are distinct settings "On Battery" or "Plugged in"...

NB.: Keep the "System cooling policy" on "Active" to avoid your PV to burn :)

If this does not help, then give a try to Throttle Stop (Basically, uncheck the BD PROCHOT checkbox an click Save)

 

Possibly, disable manually the Intel Processor Power Management feature (ippm). Run a cmd prompt as Administrator and execute this command before restarting your PC :

sc config intelppm start=disabled

NB.: To to re-enable "intelppm", execute the command

sc config intelppm start=demand

Possible values are : {boot | system | auto | demand | disabled | delayed-auto}.

 

Also look here for disabling DPTF ( "Intel(R) Dynamic Platform and Thermal Framework"). Windows has it's own built-in processor speed and thermal management already, which is sufficient on its own.

Update the drivers of your Processor, pressing Windows-R and typing: devmgmt.msc

 

Check the settings in your Bios. If you have an option "CPU Power Management" set on "ENABLED", Windows will ignore the minimum CPU power setting and clock down to the default frequency but clock up when required. If you have an option "Intel Speed Step Technology" set on "DISABLED" at the same time, the maximum frequency will be that of what it is advertised for your CPU, instead of maximum turbo boost.

Finally, you can also check your RAM just in case, by pressing Windows+R then and typing in mdsched.exe. Then restart the PC.

Finally, you should have solved the maximum Frequency:

NB.: here above, my CPU been overclocked, you see more than 100% CPU available

SynologyTips How to download playlists from YouTube using Synology

It's really easy to download playlists from YouTube with the Download Station of Synology. I do this to get locally new "No Copyright Music's", to be used later within my video's.

Click to Read More

I am using the "Download Station" as it simply works, opposite to many free or paid software's which pretend to work but usually fail (and are full of advertisements...)

Bref... Assume that you want to get all the free music's from the excellent YouTube Channel "VLOG No Copyright":

  • Visit their "PLAYLISTS" in your Browser (All such channels have a tab "PLAYLISTS" as you can see in the screenshot bellow).
  • Right-click on the link "VIEW FULL PLAYLIST" of the desired playlist.
    • This is important: don't use any URL from the "HOME" tab or from any tail representing a playlist (with the "PLAY ALL"). It would only download one video and not all the playlist. 
  • Select the menu "Copy link address"
    • The copied url should be like this : "https://www.youtube.com/playlist?list=xxxxxxxxxx"

 

Now, go to your "Synology Download Station" and:

  • Click on the large blue sign "+". It opens a window to create new download tasks.
  • Go to the tab "Enter URL".
  • Paste there the URL of the playlist copied previously.
    • A subfolder with the name of the playlist will automatically  be created under the destination folder and the video will be saved in that subfolder
  • Select the option "Show Dialog to select..." if you when to download only some of the video's. Otherwise you can unselect that option.
  • Click "Ok".

 

You should now see a line as illustrated here below, "waiting" to start:

 

You have to wait until the Download Station has crawled across the whole playlist and found all the files to be downloaded... Then, you will see the list of music's:

 

During the download, you could see some failed tasks, with a status "Error". It happens from time to time, but this is usually a temporary issue. Sort the list within the Download Station on the "Status" column and select all the "Error" . Then, right-click the list and select the menu "Resume".

 

Personally, as soon as the "music video's" are downloaded, I extract the sound tracks and keep only those. There are many software to do this. But one of my favorite option is to use ffmpeg directly on the Synology (you have to install that package). Simply type this command in a console, in the download path were are saved the video's: 

for i in *.mp4; do ffmpeg -i "$i" -codec:a libmp3lame -q:a 0 -map a "${i%.*}.mp3"; done

To process mp4 in all subfolders in the download path, I use the command:

for d in *; do cd "$PWD/$d/"; for i in *.mp4; do ffmpeg -i "$i" -codec:a libmp3lame -q:a 0 -map a "${i%.*}.mp3"; done; cd ..; done

 

If you are using Chrome, I can suggest some extensions to help in downloading Playlists..

1. The extension "Copy Selected Links". Using that one, you can copy the URL's of all the playlists of a Channel, at once. Select the whole text under the tab PLAYLISTS (it will then appear highlighted in blue as illustrated on the screenshot here under). Next, right-click in an empty area and select the menu "Copy selected Links".

Now go into Notepad++, as we have to keep only the playlists. Paste the Clipboard into a new tab of Notepad++. Press CTRL-Home to come back to the top. Press CTRL-H to open the "Replace" window. Select "Regular Expression" at the bottom of that window and click the "Replace All" button once you have entered this into the field "Find What":  .*watch.*\n (or .*watch.*\r\n if it does not work properly)

Now, you have two options. If you have no more than 50 playlists, you can paste them directly into the Download Station via the button "+" and the tab "Enter URL" as explained previously. Otherwise, you have to save with all your URL's into a .txt file (with Notepad++) and upload that file via the tab "Open a file" instead of the tab "Enter URL". 

 

2. The extension "Download Station" (or this version in Chrome Web Store) to more easily download the playlists. Same principle as above, but instead of right-clicking "View Full PlayList" to copy the playlist URL, you now have a extra menu "Synology download Station" (or Download with "Download Station").

 

Another trick, when available in the PLAYLISTS tab of a Channel: select and download the Playlists "Sorted by Mood" or "Sorted by Genre". This will help you when you search for a particular type of music for your video. Ex.: with the Channel "Audio Library":

 

Adive: each time you add a playlist in the Download Station, check that the amount of new download tasks match with the amount of tracks in the playlist. Indeed, it happens that not all the video's are added into the Download Station as it has limits(a total of max 2048 download tasks <= reason why you have to clean-up all downloaded tasks before downloading new long playlists. There is also a limit of 256 links per "file" uploaded and a limit for a playlist of "2048 - amount of current download tasks").

Don't forget that for most "No Copyright Music's", you have to mention the author in your video's or posts. Often, in YouTube, you have to click on "Show More" under the video to see the details of the license:

The License is usually clear. Ex.:

To find again later the video's on YouTube and check the type of license, it's important to keep:

  • The original name of the video when converting into mp3
  • The name of the playlist
  • The name of the channel

To do so, the easiest is to add some metadata into the mp3 extracted with ffmpeg. Once downloaded by the "Download Station", your video's should be in subfolders which have the name of their playlists. And the filenames usually contain the Author, the name of the song and the name of the YouTube Channel. So, it's easy to create the metadata. Ex.: the playlist "Dance & Electronic Music | Vlog No Copyright Music" of Vlog has been downloaded into a subfolder named "Dance & Electronic Music _ Vlog No Copyright Music" and the video's are all named like "Artist- Title (Vlog No Copyright Music).mp4"

So, to process that subfolder, I will use the command:

for i in *.mp4; do playlist=${PWD##*/}; artist=${i% - *}; other=${i##* - }; title=${other% (Vlog*}; ffmpeg -i "$i" -codec:a libmp3lame -q:a 0 -map a -metadata artist="${artist}" -metadata title="${title}" -metadata album="Playlist: ${playlist}"  -metadata Publisher="From YouTube Channel: Vlog No Copyright" "${artist} - ${title}.mp3"; done

As you can see, I am using the "Publisher" to store the Channels and the "album" for the playlist... This is a personal choice! (More details on ID3 tags and ffmpeg here)

Regarding the pattern matching used in shell, it's not always that simple and you will have to be creative... Just about the pattern matching:

  • variable2=${variable1%xyz*} => variable2 is the shortest left part of variable1 before a substring xyz.
  • variable2=${variable1%%xyz*} => variable2 is the longest left part of variable1 before a substring xyz.
  • variable2=${variable1#*xyz} => variable2 is the shortest right part of variable1 after a substring xyz.
  • variable2=${variable1##*xyz} => variable2 is the longest right part of variable1 after a substring xyz.
  • in xyz, x and z may be blanks (in my case, "xyz" was " - ")

As there are many duplicates in those playlists, I have written my own script to replace such duplicates by hardlinks (on my Synology). I can't use jdupes or rmlint as those are doing "binary comparisons" and due to the conversion via ffmpeg, it doesn't work. Instead, I search for duplicates based on the filename and size only (per channel, they are usually unique anyway). Here is my script for illustration purpose. It must be stored in an ANSI file (ex.: dedup.sh) and run with: sh dedup.sh go

echo "Start looking for duplicates"

find . -iname "*.mp3" -printf "%p ~ %f\n" | sort -f -k2 | uniq -Di -f1 > list.dup

echo "Duplicates found"

test="$1"
previous=""
previouspath=""
skip=true
while read p; do
mp3name="${p#* ~ }"
mp3path="${p%% ~ *}"
if [[ "$mp3name" == "$previous" ]]; then
mp3new="${mp3path%%.mp3}"
node=$(ls -l "$mp3path" | grep -Po '^.{11}\s1\s.*$')
if [[ "$node" == "" ]]; then
echo " $mp3path is already a hardlink"
else
skip=false
if [[ "$test" == "go" || "$test" == "test" ]]; then
SIZE1=$(stat -c%s "$mp3path")
SIZE2=$(stat -c%s "$previouspath")
#Delta=$(awk "BEGIN{ printf \"%d\n\", sqrt((100 * ($SIZE2 - $SIZE1) / $SIZE1)^2)}")
#if [[ $Delta > 1 ]]; then
Delta=$(awk "BEGIN{ printf \"%d\n\", ($SIZE2 - $SIZE1)}")
if [[ "$Delta" == "0" ]]; then
mv "$mp3path" "$mp3new.old"
ln "$previouspath" "$mp3path"
echo " $mp3path now linked to original"
else
echo " $mp3path seems different from $previouspath"
fi
else
echo " mv $mp3path $mp3new.old"
echo " ln $previouspath $mp3path"
fi
fi
else
if [[ "$test" != "test" ]] || [[ "$skip" = true ]]; then
previous=$mp3name
previouspath=$mp3path
echo "$mp3name has duplicate(s) (original in $mp3path)"
else
break
fi
fi
done <list.dup

 

Note: Facebook will possibly remove the audio from your video even if they is a "No Copyright Music". When you get such a notification from FB, you can simply click to restore the audio if you did mention the Author in your post or video.

Some YouTube Channels with "No Copyright Music's":

 

Tips How to download Adobe Pro X

I bought the version X which is now obsolete and would like to reinstall it using my license. Problem: I don't have the setup anymore... Where to download this version ?

Click to Read More

There is no link anymore on the web or on Adobe Web Site to download that version.

The trick:

  1. Log in into Adobe Web Site: https://www.adobe.com/
  2. Paste this url in your browser: http://trials3.adobe.com/AdobeProducts/APRO/10/win32/AcrobatPro_10_Web_WWEFD.exe 
  3. Run this web downloader and let it install Acrobat Pro 10

Some other obsolete stuff can be found on Adbe's FTP : ftp.adobe.com

Example:

SmartHome How to create a recursive Timer in OpenHab3's DSL Rule

I wanted a DSL Script to send a notification email if a door was still opened after several checks.

There are various options to achieve this purpose.

Click to Read More

A simple option could have been to use a Switch Item with an Expire Property. When the door opens, set that Switch Item to 'ON' and initialize a Number Item to be count how many checks will be done. When the Switch Item expires, execute a script which checks if the door is still opened. If it is not, that script ends. Otherwise, it increases the Item Number. If that Item Number is lower than the maximum checks to be done (your threshold), Set back the Switch Item to 'ON'. Otherwise, send the notification email.

Another option (just for fun) relies on a DSL Script with a lamdba expression which creates a Timer calling itself recursively until the door is closed or your threshold is reached. That DLS Script must be called when the Door opens.

Here after a Number Item SensorGarageDoor_Duration is used to count the number of checks: The Item SensorGarageDoor_OpenClose is the state of the Sensor on the Door.

val int treshold = 3
var int attempt = 0

// Define a Lambda Expression to check if the Door is open and if not,
// call itself after 15 minutes to check again until a treshold of calls is reached
val Functions$Function1 Monitor = [	Functions$Function1 recursive |
	if (SensorGarageDoor_OpenClose.state == OPEN) {
		logInfo("default.rules", "Monitor Opened Door")
		createTimerWithArgument(now.plusMinutes(15), recursive, [ Functions$Function1 callMonitor |
				var int duration = (SensorGarageDoor_Duration.state as Number).intValue + 1
				SensorGarageDoor_Duration.sendCommand(duration)
				logInfo("default.rules", "Garage Opened since " + duration.toString + " cycle(s)")
				if (duration == treshold) {
					logInfo("default.rules", "Notify by email that Garage still Opened after " + duration.toString + " cycles")
					val mailActions = getActions("mail", "your smtp binding")
					val success = mailActions.sendMail("your from", "your subject", "your message")
				} else if (duration < treshold) {
					callMonitor.apply(callMonitor)
				}
			])
	} else {
		logInfo("default.rules", "Stop monitoring Door")
	}
	true]
	
if (SensorGarageDoor_Duration.state != NULL) {
	attempt = (SensorGarageDoor_Duration.state as Number).intValue
} else {
	attempt = treshold
}

if (attempt >= treshold) {
	logInfo("default.rules", "Request for monitoring Opened Door")
	SensorGarageDoor_Duration.sendCommand(0)
	Monitor.apply(Monitor)
} else {
	logInfo("default.rules", "New request for monitoring Opened Door ignored")
}

One of the trick is to pass the Lambda Expression as parameter to the Timer (Created with createTimerWithArgument). Indeed, a Timer may not call a "Global" Lambda Expression. To get a Lambda Expression as parameter, this parameter must be defined as Functions$FunctionX (where X is the amount of parameters of the Lambda Expression). In my sample above, my Lambda as one parameter which is itself, to be passed to the Trigger (So, it is typed Functions$Function1).

You must type explicitly the parameters as Functions$Function1 to be able to use the method "apply()".

Raspberry Pi Xiamo Aqara and Openhab on Raspberry Pi

A few notes on using Xiaomi Aqara Sensors and Switches with Openhab 3.

Click to Read More

I am using a Zigbee USB key Conbee II from Dresen Eletronik.

It is plugged into the Raspberry Pi with a USB cable extension to put it away from the other keys used on this RPi, like my Sigma Z-Wave controller.

I am sing OpenHab 3 (previously OpenHab 2).

Opposite to the Sigma Z-Wave USB key, there is not mount issue after a reboot due to the usb port id increasing... (See also a solution in French).

I am using the Phoscon App to connect and manage devices via the Conbee II key. The App is accessible on the http port 80 of the Raspberry.

To connect a new Xiaomi Aqara device,

  1. "Add a Sensor" or a "Add a Switch" within Phoscon. Pick the type "Other".
  2. Press the reset button for 5 sec. The blue led should turn on and start blinking after 5 sec. Release the button immediately when it starts blinking. The led will then turn on for 2 or 3 sec and then blink 3 times. It means that the connection is established.
  3. A few second later, the device will appear in Phoscon.

To get the Things into Openhab, Add the Binding "Dresen Eletronik DeCONZ Binding" and "Scan" for devices.

I am using with success:

  • Multi Sensors : with values for Temperature, Humidity and Pressure
  • Door and Window Sensors: with Status Opened / Closed
  • A Cube : with events Pushed (various direction), Rotation, Shacked
  • Wireless Mini Switch : One Button Control with 4 events: Single Press, Double Press, Long Press and Long Press Release

 

Notice: 

  • after a few month using the Door Sensors on my Garage Door, it stopped to send events... The problem was that the captor part was attached on a metal upright which ended to be magnetized by the magnet part moving around...

Raspberry Pi USB ports used by openHab's Z-Wave Controller change after each reboot on RPI

Usually, USB keys get assigned a new port such as /dev/ttyACM0, /dev/ttyACM1, etc..., each time they are unplugged and replugged into the RPI or if the RPI reboots. A solution consists in making these ports permanent via symlinks.

Click to Read More

This "feature" is a problem as the specified device may not be recognized anymore by configured softwares, like the Z-Wave Binding or openHab.

See a definitive solution here: make serial USB ports persistent via symlinks.

Synology Sudoer file not working on Synology due to dots in its name

I spent one hour to investigate why I couldn't execute a command with sudo, from a php script, although the user was authorized for that command within a sudoer file... The problem was a dot in the name of the sudoer file.

Click to Read More

My php script is part of a package I have created to run on my Synology (DSM 7.x).. It is running under an account named like my package: MODS_Package7.x

That php script executes the following code:

$COMMAND = "sudo /usr/syno/bin/synopkg start '$PACKAGE' 2>&1";
exec($COMMAND, $output, $result);

My sudoer file was named /etc/sudoers.d/MODS_Package7.x and contained:

MODS_Package7.x ALL=(ALL) NOPASSWD: /usr/syno/bin/synopkg

It didn't work until I removed the ".", renaming the sudoer file into /etc/sudoers.d/MODS_Package7_x

 

How stupid,  but it's indeed mentioned in the documentation:

sudo will read each file in /etc/sudoers.d, skipping file names that end in ‘~' or contain a ‘.' character to avoid causing problems with package manager or editor temporary/backup files.

The /etc/sudoers.d/README file does not exist on Synology, but can be found on other Linux


#
# As of Debian version 1.7.2p1-1, the default /etc/sudoers file created on
# installation of the package now includes the directive:
# 
#   #includedir /etc/sudoers.d
# 
# This will cause sudo to read and parse any files in the /etc/sudoers.d 
# directory that do not end in '~' or contain a '.' character.
# 
# Note that there must be at least one file in the sudoers.d directory (this
# one will do), and all files in this directory should be mode 0440.
# 
# Note also, that because sudoers contents can vary widely, no attempt is 
# made to add this directive to existing sudoers files on upgrade.  Feel free
# to add the above directive to the end of your /etc/sudoers file to enable 
# this functionality for existing installations if you wish!
#
# Finally, please note that using the visudo command is the recommended way
# to update sudoers content, since it protects against many failure modes.
# See the man page for visudo for more information.
#

 

Synology Run a command as root on Synology with any user

Synology is reinforcing the security within its DMS, making more difficult the execution of scripts as root from packages. Here is my trick to do so based on the use of ssh and php.

Click to Read More

Theoretically, to run something as root, a user must be a sudoer, i.e.: a user having the right to execute commands as root. It's not difficult to configure a user to be a sudoer. You simply have to add that user into a file located under /etc/sudoers.d/, with a few parameters to describe what he can execute and if he requires to type his password or not. Ex.: to let a user named "beatificabytes" able to execute the command 'shutdown' on a Synology:

  1. Create a file (the name does not matter): /etc/sudoers.d/beatificabytes 
  2. Edit that file with vi (or nano if you installed that package). Notice that you should really be careful as any error in that file will prevent you to log in anymore:  DANGER !!! That's why it is usually highly recommended to use the command 'visudo' to edit such files (it checks the syntax before saving changes)... Unfortunately, this command is not available on Synology.
  3. To grant rights without having to type a password, type this: beatificabytes ALL=(ALL) NOPASSWD: /sbin/shutdown

The problem I have with this approach is that I have many various users (each package runs with its own user) and I don't want to define them all as sudoers.

One option would be to run all packages with the same user who is defined as a sudoer. This is possible via a privilege file to be added into the packages (/conf/privilege).

But if like me you want to always use the same administrator account, who is already a sudoer, if you have php installed on your NAS and if you have enabled ssh, then your could simply open a ssh session with that admin account, from a php script, to execute your commands.

Here is the php script I am using for such a purpose (saved into a file named 'sudo.php'):

<?php
$options = getopt("u:p:s:o:c:");
$user = $options['u'];
$password = $options['p'];
$server = $options['s'];
$port = $options['o'];
$command = $options['c'];

if (!function_exists("ssh2_connect")) die("php module ssh2.so not loaded");
if(!($con = ssh2_connect($server, $port))){
    echo "fail: unable to establish connection to '$server:$port'\n";
} else {
    // try to authenticate with username password
    if(!ssh2_auth_password($con, $user, $password)) {
	if (strlen($password) > 2){
		$pass = str_repeat("*",strlen($password)-2);
	} else {
		$pass = "";
	}
	$pass =  substr($password, 0,1).$pass.substr($password, -1);
        echo "fail: unable to authenticate with user '$user' and password '$pass'\n";
    } else {
        // allright, we're in!
        echo "okay: logged in...\n";

        // execute a command
	$command = "echo '$password' | sudo -S $command 2>&1 ";
        if (!($stream = ssh2_exec($con, $command ))) {
            echo "fail: unable to execute command '$command'\n";
        } else {
            // collect returning data from command
            stream_set_blocking($stream, true);
            $data = "";
            while ($buf = fread($stream,4096)) {
                $data .= $buf;
		echo $buf;
            }
            fclose($stream);
        }
    }
}
?>

And to call it, from a shell for example, assuming that your are using php7.3, type something like:

php -dextension=/volume1/@appstore/PHP7.3/usr/local/lib/php73/modules/ssh2.so sudo.php -u YourAdminAccount -p YourAdminPassword -s 127.0.0.1 -o 22 -c "whoami"

Here above

  • 22 is the port defined for ssh on my Synology
  • I executed the command 'whoami'. So, the outcome will be "root"

Et voilà.

Tips Low dialog volume with DTS movies on RealTek compared to AC3

A simple trick to fix the low dialog volumes of a DTS audio track played with a Realtek is to use the Loudness Equalization.

Click to Read More

I am using Home Media Player Classic x64 v1.9.8 (24-10-2020) to play movies on my Windows 10.  My Motherboard is a Z270 Gaming Pro Carbon with a Realtek® ALC1220 Codec.

When playing movies with a DTS audio track, the volume is always really low compared to movies with an AC3 audio track, especially regarding the dialogs.

It was not the case when I initially installed Windows so I presume that this is a regression due to some driver updates... But a simple trick was to enable the Loudness Equalization via the Realtek HD Audio Manager.

If you don't know how to open the Realtek HD Audio Manager, open it directly from Windows Explorer or with Win+R: C:\Program Files\Realtek\Audio\HDA\RtkNGUI64.exe

[EDIT 23/05/2021] This trick didn't solve the issues with Netflix. The voices were really inaudible. I had to switch the sound of Netflix from 5.1 to 2.0. I am sure there is another solution. But I didn't find it yet.

Et voilà,