Terminal Takeaway 🥡

rwaltr · April 12, 2021, 12:15am

Ulfnic · April 12, 2021, 10:09pm

I probably should take a look at fd, i’ve heard it mentioned more than once and I really appreciate recommendations. It’s not enough for something to seem better, I like to know smart people are using it. Kind of like how you can find the best restaurants when you’re traveling by checking how many locals are sitting down.

Then there’s the tyranny of the defaults…

If I make a guide that uses fd, then it becomes a dependency I need to mention as it’ll require an extra step. I also have to anticipate it whenever a I build a new system for myself or someone else that’ll use that script. The script will no longer run out-of-the-box, it’s loosely the same reason people try to stay POSIX compliant even at a performance hit because it can be (generally speaking) depended on to work in a wide range of environments.

Then there’s the advancement problem…

If no one moves to fd it won’t become standardized enough for distros to adopt it as a core package. It’s one of the reasons i’ve come to love Fedora as they’ll push those boundaries but not too fast like what they’re doing in PipeWire.

Then there’s the learning investment…

Should I learn something that hasn’t see mass adoption yet and may never if it’s replaced with something better. That’s a personal question because sometimes the performance is critical, other times not so much. It adds more trust requirement because now there’s more programmers managing my system but I wouldn’t have a system if I didn’t pass out a degree of trust to begin with. A lot of pros an cons…

So to fd or not fd… THAT! is the question.

rwaltr · April 13, 2021, 4:25pm

its not a one to one replacment, but its speed alone makes it a good alt

Ulfnic · April 20, 2021, 4:12pm

Unveiling your shell’s secrets

Using shell variables can be a bit like using dark magic… there’s always one you didn’t know about. Here’s one that contains your last directory:

$ pwd
/usr/bin
$ cd /etc
$ pwd
/etc
$ echo $OLDPWD
/usr/bin

`printenv`

printenv can shed some light. Without arguments it’ll print the value pairs of every environment variable in the shell. You can also see just one by using printenv <variable>.

Using | wc -l to count the value pairs I have 43 but OLDPWD is missing.

$ printenv | wc -l
43
$ printenv OLDPWD

`declare`

declare -p will list EVERY value pair including what printenv does. Here I get 94 and OLDPWD’s included.

$ declare -p | wc -l
94
$ declare -p OLDPWD
declare -x OLDPWD

But what about the functions?

declare can also show every function with -f for the whole function and -F for just the name. Be aware you can’t combine the arguments to get both lists.

I have quite a few custom functions in my .bashrc so I get 115. declare -p && declare -f shows me everything.

$ (declare -p && declare -F) | wc -l
115
$ declare -p && declare -f
(Everything...)

Did you know your shell had functions __vte_osc7 and __vte_prompt_command ? I sure didn’t.

MarkofCain · April 20, 2021, 6:42pm

This is some good info. I didn’t know about the functions either. Allow me to add a shortcut for $OLDPWD

“-” calls the $OLDPWD as in:

$ pwd
/bar/foo/folder/buried/deep/in/your/data/organizational/mine/field

$ cd /etc
$ pwd
/etc

$ cd -
$ pwd 
/bar/foo/folder/buried/deep/in/your/data/organizational/mine/field

Ulfnic · April 20, 2021, 7:34pm

Didn’t know about cd -, nice!

jastombaugh · April 23, 2021, 6:22pm

Can I just put out here that I ABSOLUTELY LOVE THIS THREAD!!!

Thank you so much for this compilation of delightful tips and tricks. Every time I stop in here, I learn something fun, helpful, or revolutionary.

Thank you!

Ulfnic · May 2, 2021, 9:55pm

ARG!! Where do I put my executable???

Asking your distro’s opinion… $PATH contains a : deliminated string of each directory that’s checked for executables and the directory names reveal if they’re managed and who they’re intended for.

Keep in mind every distro/system can be different though the terminology is often shared, this example is for Fedora:

# Try it:

$ echo $PATH
/home/ulfnic/.local/bin:/home/ulfnic/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

# Easier to read:
$ echo $PATH | tr ':' '\n'
# Or
$ echo $PATH | sed 's/:/\n/g'
/home/ulfnic/.local/bin
/home/ulfnic/bin
/usr/local/bin
/usr/bin
/usr/local/sbin
/usr/sbin

If I attempt to run xcalc (the calculator 99.99% of Linux users are surprised to learn they have installed), the system will check the current directory, then those 6 directories from top to bottom till it finds xcalc and runs it.

Where is xcalc?

$ which xcalc
/usr/bin/xcalc

Should I uninstall xcalc?
That’s a philiosophical question beyond the scope of this guide.

Terminology

Everyone describes these directories a bit differently but this is my practical summary:

local is for files that aren’t ordinarily managed by the system (with things like dnf or apt). It’s a safe place for files to not be overwritten during standard installations and updates.

It’s commonly refered to as the place for “local” or “unique” files but syncing them against a public/private repo is just as valid a use case.
bin is short for binary but all executables are welcome.
sbin is like bin but intended for the superuser only.
usr is for all users.
home is for specific users.

Breaking it down

/home/ulfnic/.local/bin ← I’m a custom executable intended for Ulfnic
/home/ulfnic/bin ← I’m a managed executable intended for Ulfnic
/usr/local/bin ← I’m a custom executable intended for everyone
/usr/bin ← I’m a managed executable intended for everyone
/usr/local/sbin ← I’m a custom executable intended for the superuser
/usr/sbin ← I’m a managed executable intended for the superuser

File & folder permissions ultimately determine who has access to what, for example if I chmod +x instead of chmod u+x a file in my user folder, all users will be able to execute it but the above is a decent guide to what these folders are generally intended for.

Where should I put my executable?

For a specific user: /home/$USER/.local/bin
For all users: /usr/local/bin
For superuser: /usr/local/sbin

BONUS: Making a shim

Because /local/bin directories are checked prior to their /bin equivalents it makes it easy to run a script first that then runs the program being asked for.

Example:

$ which nano # Make sure you know where the nano binary is first.
/usr/bin/nano

$ sudo touch /usr/local/bin/nano # Make a file named nano in /usr/local/bin/ so it's found first and run instead of the real nano.

$ sudo nano /usr/local/bin/nano # Edit
# Paste the following:

#!/usr/bin/env sh
echo "vim empowers users by teaching them lots of useful key"
echo "commands found in other terminal applications!"
echo ""
read -r -p "Are you sure you only want to learn nano? [y/N] " Res
case "$Res" in
	[yY][eE][sS]|[yY])
		# Respect the user's decision, Linux is about choice.
		# Make sure to use nano's direct path gathered from `which nano` or this script will loop.
		# Appending "$@" includes all the arguments so `nano myfile.txt` will still work.
		/usr/bin/nano "$@"
	;;
	*)
		# Check if vim is installed.
		if $( -z which vim 2>/dev/null); then
			echo
			echo "Your distro is afraid even the GPL won't protect"
			echo "them from the liability of users getting stuck in VIM!"
			echo "You need to install it at your own risk."
		else
			echo
			echo "Godspeed."
			sleep 1
			echo "Wait! Esc :q!"
			sleep 1
			vim "$@" # Transcend the user.
		fi
	;;
esac

# Save & exit (Ctrl + x)
$ sudo chmod +x /usr/local/bin/nano # Make it executable to all users

# The terminal session may need to be refreshed to see the change
$ exec bash

# Test the shim
$ nano

# Remove the shim in compliance with HR
$ sudo rm /usr/local/bin/nano

jastombaugh · May 3, 2021, 1:58am

HAHAHAAAA!!! Nice!

Ulfnic · May 8, 2021, 5:13am

Some examples of PCRE RegEx with inline Perl:

#!/usr/bin/env sh

# Define values for examples
$ REGEX="23"
$ VAL="1234"
$ ADD_VAL="+"

# Insert ADD_VAL before the RegEx match
$ perl -e "'$VAL' =~ m/$REGEX/s; print \"$\`$ADD_VAL$&$'\";"
1+234

# Insert ADD_VAL after the RegEx match
$ perl -e "'$VAL' =~ m/$REGEX/s; print \"$\`$&$ADD_VAL$'\";"
123+4

# Replace the RegEx match with ADD_VAL
$ perl -e "'$VAL' =~ m/$REGEX/s; print \"$\`$ADD_VAL$'\";"
1+4

# Remove RegEx match from VAL
$ perl -e "'$VAL' =~ m/$REGEX/s; print \"$\`$'\";"
14

# Output RegEx match within VAL
$ perl -e "'$VAL' =~ m/$REGEX/s; print \"$&\";"
23

Ulfnic · June 1, 2021, 2:54am

The Linux Foundation: Filesystem Hierarchy Standard

The definitive guide to, “Where the hell do I put this?” and, “Why the hell is this put here?”

Filesystem Hierarchy Standard

Ulfnic · June 6, 2021, 1:57am

Simple password based AES-256 symmetric encryption using gpg:

Try it:

# Make a file
echo 'pkgs.org is a distro search engine for packages!' > 1-toencrypt.txt
# Encrypt
gpg -cao 2-encrypted.txt --cipher-algo AES256 --no-symkey-cache 1-toencrypt.txt
# Decrypt
gpg -o 3-unencrypted.txt --no-symkey-cache 2-encrypted.txt

Result:

├─ 1-toencrypt.txt
├─ 2-encrypted.txt
└─ 3-unencrypted.txt

Break down:

# Make a file containing a secret
echo 'pkgs.org is a distro search engine for packages!' > toencrypt.txt

# Encrypt toencrypt.txt using AES256
gpg -cao encrypted.txt --cipher-algo AES256 --no-symkey-cache toencrypt.txt
	# -c | --symmetric, Encrypt with a symmetric cipher using a passphrase.
	# -a | --armor, use ASCII "armored" output instead of binary (useful for Email, Web POST, ect)
	# -o | --output <file>, output file (default: input file name with ".asc" appended)
	# --cipher-algo <algorithm>, which algorithm to use (default: AES128)
		# Bonus: See which algorithms `gpg` supports by using `gpg --version`
	# --no-symkey-cache, disable gpg's default behavior of caching passwords

# Enter password at prompts

# Output encrypted file
cat encrypted.txt

-----BEGIN PGP MESSAGE-----

jA0ECQMCBXVoK9eD6ar/0nYBBz0B2EX0Z7R6UaOjjwDUNl3TublbeCy7rlJ7j6+t
Nqmvbm1wgmqsY/eY8171wMDInaa1M/VAoRFSIDRTAkx798ymbbsx/CUJakRH3Kll
kpWG1UfyDqUgWACOgpQxvjZyA4qhlruvYuI7IXeb0KEskFUEMdeQ
=33Fs
-----END PGP MESSAGE-----

# Unencrypt
gpg -o unencrypted.txt --no-symkey-cache encrypted.txt
	# -o | --output <file>
	# --no-symkey-cache, disable gpg's default behavior of caching passwords

# Enter password at prompt

# Output unencrypted file
cat unencrypted.txt

pkgs.org is a distro search engine for packages!

Bonus: Clearing `gpg-agent` password cache

Without the --no-symkey-cache argument above, passwords are cached by the gpg-agent allowing decryption of those files without a password until the user logs out.

The password manager pass also has gpg-agent cache the master password in the same way so this might be useful to pass users too.

To remove all cached passwords without having to logout, use the following command:

gpgconf --reload gpg-agent

Ulfnic · June 12, 2021, 3:55am

Why the name change?

There seems to already be various other Terminal Tuesdays which came first so this is squeezing in on other people’s namespace.
There’s a hidden “rule 0” joke that I made this on a Monday and almost never posted on Tuesday but it was a bit too stealthy and is more likely to create confusion.
It’d be nice to have a domain that isn’t taken for the thread

Without further ado! Welcome to Terminal Takeaway

vinylninja · June 12, 2021, 8:36pm

It makes more sense and I think I never really posted something on Tuesday either. Every day should be a terminal day.

Ulfnic · June 17, 2021, 6:50pm

“Arrays” in POSIX

POSIX Shell doesn’t have arrays but it does have “an array”… kind of.

You can use the tools for managing arguments in semi-familiar ways to how arrays are managed in BASH. Each function has it’s own arguments too so each one has it’s own “array”.

Play in terminal:

# Set arguments (do this first if playing in terminal)
set Setting 5 new array "elements here"

# Append an argument
set -- "$@" "and 1 more"

# Print number of arguments
echo $#

# Print all arguments (argument spaces and separator spaces are separate)
echo $@

# Print all arguments (spaces are ambiguous)
echo $*

# Print the 2nd argument
# Note: After $9 braces should be used to assure cross shell compatibility, ie: ${10}
echo $2

# Remove 2 arguments from the beginning
shift 2

# Remove 2 arguments from the end using rev
set -- $(rev <<< "$@")
shift 2
set -- $(rev <<< "$@")

# Remove 2 arguments from the end cycling through a loop
for i in $(seq 1 $#); do
	[ "$i" -le $(($# - 2)) ] && set -- "$@" "$1"
	shift
done

# Remove all arguements
shift $#

# Loop through all arguments
for Arg do
	echo "$Arg"
done

Ulfnic · June 28, 2021, 2:31am

Running an app as a different user in the same X session:

# Create a test user account for the example and give it a password
sudo useradd testdummy
sudo passwd testdummy

# Option 1. Using su (Switch User)
su - testdummy -c "DISPLAY=:0 /usr/bin/firefox"

# Option 2. Using runuser
sudo runuser -u "testdummy" -- /usr/bin/firefox

# Cleanup: Remove the test account and it's home directory
sudo userdel -r testdummy

Useful for running apps in different configurations if they don’t support profiles (which Firefox does) and sending notify-send alerts from scripts run with sudo, ex:

runuser -u "ulfnic" -- notify-send "Backup complete!" -t 0

“The difference between the commands runuser and su is that runuser does not ask for a password (because it may be executed by the root user only) and it uses a different PAM configuration. The command runuser does not have to be installed with set-user-ID permissions.”

https://www.linux.org/docs/man1/runuser.html

Ulfnic · July 4, 2021, 9:06pm

Controlling Keyboard LEDs

Try it:

xset led named "Scroll Lock"
xset -led named "Scroll Lock"

Behavior:

xset won’t cause the system to forget or mismatch the organic state of the LED. Turning an LED “off” with xset just returns it to it’s organic state so if it was originally on before turning it “on” then it’ll stay on.
xset doesn’t activate the mode the LED is associated with. *

Going beyond Scroll Lock:

So what about the other LEDs?? This command will turn them all on…

# 1. Turn on the Christmas Tree!
xset led on

# 2. Express disappointment

# 3. Check if it did anything
xset q

Keyboard Control:
  auto repeat:  on    key click percent:  0    LED mask:  ffffe7fc
  XKB indicators:
    00: Caps Lock:   off    01: Num Lock:    off    02: Scroll Lock: on 
    03: Compose:     on     04: Kana:        on     05: Sleep:       on 
    06: Suspend:     on     07: Mute:        on     08: Misc:        on 
    09: Mail:        on     10: Charging:    on     11: Shift Lock:  off
    12: Group 2:     off    13: Mouse Keys:  on

# 4. Switch LEDs off
xset led off

* It turns out most distros consider Caps Lock and Num Lock LEDs forbidden fruit and for good reason. Below is a guide to allow you to use them but (at least on my machine) they reliably produce unpredictable keyboard results such as the Enter key being held down, Shift being held down, being totally locked out of the numlock keys and sometimes all of these behaviors stick around even after the LEDs are turned off.

Scroll lock appears to be the only “safe” LED compared to the fore-mentioned. Caps Lock is usually ok and Num Lock opens the door to the chaos dimension.

# Enabling Caps Lock and Num Lock LEDs (caution, see above ^)
# Guide applies to: Debian, Ubuntu, Fedora and CentOS

# === Option 1. Manually adjust: ===
# Caps Lock:
sudo nano /usr/share/X11/xkb/compat/ledcaps
# Change: !allowExplicit; to allowExplicit;

# Num Lock:
sudo nano /usr/share/X11/xkb/compat/lednum
# Change: !allowExplicit; to allowExplicit;

# === Option 2. Auto-adjust: ===
# Caps Lock:
sudo sed -i 's|\!allowExplicit|allowExplicit|g' /usr/share/X11/xkb/compat/ledcaps

# Num Lock:
sudo sed -i 's|\!allowExplicit|allowExplicit|g' /usr/share/X11/xkb/compat/lednum

# Logout and back in

# Test
xset led named "Caps Lock"
xset -led named "Caps Lock"
xset led named "Num Lock"
xset -led named "Num Lock"

Scroll Lock LED alarm script:

cd /usr/local/bin
sudo touch ./sl-alarm
sudo chmod 755 ./sl-alarm
sudo nano ./sl-alarm
# Paste the following:

#!/usr/bin/env sh

# If the application is terminated, set Scroll Lock LED off
trap 'xset -led named "Scroll Lock"' EXIT

while true; do
	xset led named "Scroll Lock"
	sleep 0.4
	xset -led named "Scroll Lock"
	sleep 0.4
done

# Save & quit
# Run
sl-alarm
# Ctrl + c to quit

Extra:

You can also activate LEDs using their id from xset q + 1

# Example:
xset led 1 # Caps Lock
xset led 2 # Num Lock
xset led 3 # Scroll Lock

Ulfnic · July 9, 2021, 12:38am

Does anyone know of a way to send keystrokes to an X window? Other than…

xdotool type "something"
xdotool key "a"
xvkbd -text "something"

List:

xdotool
xvkbd
crikey [website] - thanks to Ethanol

Ethanol · July 9, 2021, 6:47am

This may not be exactly what you’re looking for but…

I know a program called “crikey”

https://shallowsky.com/software/crikey/

It’s used for outputting a string with a hotkey wherever the cursor is. You can pipe stuff into it as well.
Example:
echo "hello" | crikey -i -s 2

This will print “hello” after two seconds to the active window. I use this with zbarcam to get raw barcode data from a webcam and type it into a database form input.

Example:
zbarcam /dev/video0 --raw | crikey -i

Ulfnic · July 9, 2021, 7:18am

That’s a good tool to know about.

Ironically one of the things that makes it so cool (handling of special codes) makes it not work for my project although getting more options was sort of a quest for a silver bullet anyway and that’s quickly been turning into the realization I need to do a lot of custom handling to make this work.

I’ve been putting off the description because It’s solvable with a single line of code unless I start describing the 1,000 edge cases :P. I’ll post it soon.

Terminal Takeaway 🥡

Unveiling your shell’s secrets

printenv

declare

ARG!! Where do I put my executable???

Terminology

Breaking it down

Where should I put my executable?

BONUS: Making a shim

The Linux Foundation: Filesystem Hierarchy Standard

Simple password based AES-256 symmetric encryption using gpg:

Try it:

Break down:

Bonus: Clearing gpg-agent password cache

“Arrays” in POSIX

Play in terminal:

Controlling Keyboard LEDs

Try it:

Going beyond Scroll Lock:

Scroll Lock LED alarm script:

Extra:

List:

`printenv`

`declare`

Bonus: Clearing `gpg-agent` password cache