fabrice/scripts-fabq
fabrice/scripts-fabq
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(notes): expand storage, btrfs, linux, and ssh notes

Browse Source 
btrfs.md:
- Add device scan (no-arg form) and device stats --reset / -z
- Add read-only mount, remount with performance/default options
- Add Degraded Mount and Missing Device Removal subsection
- Add stdbuf + zstd backgrounded defrag with log output
- Add RAID1 balance conversion
- Add ionice for scrub, watch scrub+device stats, watch scrub+temperatures
- Add Recovery → Filesystem Check (btrfs check, --force)
- Add Recovery → Diagnosis (journal by date, dmesg btrfs filter)

linux.md:
- Add CPU info commands (lscpu, /proc/cpuinfo, core count)
- Add GPU info (lspci | grep vga) and CPU scaling driver
- Add mount --bind and chroot with pseudo-fs setup
- Add update-initramfs -k all and -c -k $(uname -r) variants
- Add NFS section (showmount, exportfs)
- Add Network Diagnostics section (curl timing, high-freq ping, jumbo frame ping)
- Add journal date filtering and microcode grep to Diagnosis

ssh.md:
- Add Placeholders section
- Replace all hardcoded usernames, hostnames, IPs, and key paths with placeholders
- Add Skip Host Key Verification section (StrictHostKeyChecking, UserKnownHostsFile)

storage.md:
- Add lsblk -e 7 to exclude loop devices
- Add blkid <partition> variant
- Add cat /proc/mounts grep
- Add SMART filtered greps (Reallocated, Pending, UDMA_CRC, Load_Cycle_Count)
- Add drive temperature one-liner and watch loop
- Add Hardware Monitoring section (lm-sensors, sensors-detect, sensors)
- Add Kernel Messages section (dmesg tail, err/warn, -T, nvme, ata/scsi, I/O, ATA port mapping)
- Add strings on raw device to Hex Dump
This commit is contained in:
Fabrice Quenneville 2026-05-12 01:03:26 -04:00
parent cd4cf23f92
commit 89e4bd519a
4 changed files with 667 additions and 193 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

162
notes/btrfs.md
View File

@ -9,6 +9,7 @@
- [Information on Filesystem](#information-on-filesystem)
- [Drive Manipulation](#drive-manipulation)
- [Replace Drives](#replace-drives)
- [Degraded Mount and Missing Device Removal](#degraded-mount-and-missing-device-removal)
- [Filesystem Manipulation](#filesystem-manipulation)
- [Upgrading Btrfs Block Group Cache to V2](#upgrading-btrfs-block-group-cache-to-v2)
- [Defrag](#defrag)
@ -19,6 +20,8 @@
- [Delete Snapshots](#delete-snapshots)
- [Backup Procedures](#backup-procedures)
- [Recovery](#recovery)
- [Filesystem Check](#filesystem-check)
- [Diagnosis](#diagnosis)
## Placeholders
@ -110,6 +113,7 @@ btrfs device usage <mountpoint>
Scan all devices or a specific drive:
```bash
btrfs device scan
btrfs device scan <device>
```
@ -121,6 +125,17 @@ Read errors, write errors, flush errors, etc.:
btrfs device stats <mountpoint>
```
**Reset Device Error Counters**
Reset all per-device error counters to zero after acknowledging them:
```bash
btrfs device stats --reset <mountpoint>
btrfs device stats -z <mountpoint>
```
- `-z` / `--reset`: Zeroes the counters after printing. Useful after a known event you've already investigated.
**List BTRFS Subvolumes**
```bash
@ -179,6 +194,28 @@ btrfs subvol list /
mount -o subvolid=<subvolume-id> /dev/disk/by-uuid/<uuid> <mountpoint>
```
**Mount Read-Only**
Mount a partition in read-only mode, useful for forensics or recovery without risking further writes:
```bash
mount -r <device> <mountpoint>
```
**Remount with Performance Options**
Apply common performance mount options to a live filesystem without unmounting:
```bash
mount -o remount,noatime,compress=zstd:3,autodefrag,space_cache=v2 <mountpoint>
```
**Remount with Default Options**
```bash
mount -o remount,defaults,noatime,compress=zstd:3 <mountpoint>
```
**Add a New Drive**
```bash
@ -233,6 +270,33 @@ btrfs replace status -i <mountpoint>
- Useful for both failing drive replacement and capacity upgrades.
- Ensure the target drive has enough space to accommodate the source data.
### Degraded Mount and Missing Device Removal
Use when a drive has failed and you need to access the filesystem with the remaining devices.
**Mount in degraded mode:**
```bash
mount -o ro,degraded <device> <mountpoint>
```
**Mount a specific subvolume in degraded mode:**
```bash
mount -t btrfs -o degraded,subvol=<subvolume>,noatime,compress=zstd:3 UUID=<uuid> <mountpoint>
```
**Remove the missing device from the filesystem:**
Once mounted degraded, remove the placeholder for the missing drive:
```bash
btrfs device remove missing <mountpoint>
```
- This cleans up the missing device slot so the filesystem no longer expects it.
- Only safe to run if data is intact on the remaining devices (e.g., RAID1 with one drive).
## Filesystem Manipulation
### Upgrading Btrfs Block Group Cache to V2
@ -298,6 +362,8 @@ umount <mountpoint>
### Defrag
**Standard recursive defrag with LZO compression:**
```bash
btrfs filesystem defrag -r -v -clzo <mountpoint>
```
@ -306,6 +372,18 @@ btrfs filesystem defrag -r -v -clzo <mountpoint>
- `-v`: Verbose.
- `-clzo`: Optional LZO compression to save space.
**Recursive defrag with Zstd compression, logged to file:**
Runs in the background with unbuffered output so the log file updates in real time:
```bash
stdbuf -oL btrfs filesystem defrag -r -v -czstd <mountpoint> > /root/<date>-defrag.log 2>&1 &
```
- `stdbuf -oL`: Forces line-buffered stdout so log entries appear immediately.
- `-czstd`: Zstd compression (better ratio than LZO, available since kernel 5.1).
- `&`: Runs in the background; use `tail -f /root/<date>-defrag.log` to monitor.
## Balances
**Full balance on nearly empty block groups:**
@ -351,6 +429,14 @@ btrfs balance start --bg --full-balance -dusage=0 -musage=0 <mountpoint>
btrfs balance start --bg -dlimit=100 <mountpoint>
```
**Convert to RAID1:**
Rebalances data and metadata to RAID1 profile. Use after adding a second drive or to switch from single to mirrored:
```bash
btrfs balance start -mconvert=raid1 -dconvert=raid1 <mountpoint>
```
**Cancel a balance:**
```bash
@ -367,7 +453,7 @@ btrfs balance status <mountpoint>
**Start a scrub**
The scrub operation verifies data integrity against checksums
The scrub operation verifies data integrity against checksums:
```bash
btrfs scrub start <mountpoint>
@ -385,6 +471,37 @@ btrfs scrub status <mountpoint>
btrfs scrub cancel <mountpoint>
```
**Lower scrub I/O priority:**
Reduce the impact of a running scrub on system I/O by setting it to idle class:
```bash
ionice -c 3 -p $(pgrep btrfs-scrub)
```
- `-c 3`: Idle class — only uses I/O when no other process needs it.
**Watch scrub status and device stats:**
Continuously display scrub progress and per-device error counters:
```bash
watch -n 10 "btrfs scrub status <mountpoint>; echo ''; btrfs device stats <mountpoint>"
```
**Watch scrub status and all drive temperatures:**
```bash
watch -n 5 "btrfs scrub status <mountpoint> && echo '' && \
smartctl --scan | awk '{print \$1}' | while read dev; do \
echo -n \"\$dev: \"; \
smartctl -A \$dev | grep -iE 'Temperature|Airflow_Temp' | awk '\
/Temperature_Celsius/ {print \$10 \"°C\"} \
/Airflow_Temperature_Cel/ {print \$10 \"°C\"} \
/Temperature:/ {print \$2 \"°C\"}' | head -n 1; \
done && echo '' && btrfs device stats <mountpoint>"
```
## Snapshots
### Create Snapshots
@ -476,3 +593,46 @@ btrfs scrub cancel <mountpoint>
```bash
btrfs restore -D <device>
```
### Filesystem Check
Run offline consistency checks on an unmounted BTRFS filesystem.
**Check an unmounted filesystem:**
```bash
btrfs check <device>
```
- Must be run on an **unmounted** device. Running on a mounted filesystem risks corruption.
- Use the UUID path if needed: `/dev/disk/by-uuid/<uuid>`
**Force check (use with caution):**
```bash
btrfs check --force <device>
```
- `--force`: Bypasses the mount check. Only use this if you are certain the filesystem is not mounted and understand the risks.
### Diagnosis
Filter system logs and kernel messages to diagnose BTRFS-related events.
**Search journal logs by date range:**
```bash
journalctl --since "<date>" --until "<date>" | grep -i btrfs
```
Example:
```bash
journalctl --since "2026-01-01" --until "2026-01-02" | grep -i btrfs
```
**Search kernel ring buffer for BTRFS events:**
```bash
dmesg | grep -i btrfs
```

158
notes/linux.md
View File

@ -14,6 +14,8 @@
- [System Management](#system-management)
- [Change password of a tar/openssl archive](#change-password-of-a-taropenssl-archive)
- [Verify two possibly identical folders recursively](#verify-two-possibly-identical-folders-recursively)
- [NFS](#nfs)
- [Network Diagnostics](#network-diagnostics)
- [Diagnosis](#diagnosis)
- [Debian Upgrade Issues](#debian-upgrade-issues)
- [Wayland Issues](#wayland-issues)
@ -35,6 +37,32 @@ To gather detailed information about your hardware, use the following commands:
- `dmidecode -t memory` for RAM details
- `dmidecode -t bios` for BIOS information
**CPU information**
```bash
lscpu
cat /proc/cpuinfo
grep -c 'model name' /proc/cpuinfo
```
- `lscpu`: Structured summary of CPU architecture, cores, threads, and NUMA topology.
- `cat /proc/cpuinfo`: Raw per-core details including model name, flags, and frequencies.
- `grep -c 'model name'`: Quick count of logical CPU cores.
**GPU information**
```bash
lspci | grep -i vga
```
**CPU frequency scaling driver**
Check which driver is managing CPU frequency scaling (e.g., `intel_pstate`, `acpi-cpufreq`):
```bash
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver
```
### Software Information
**Finding information on the Linux distribution**
@ -257,6 +285,60 @@ ln -s /usr/share/zoneinfo/<timezone> /etc/localtime
systemctl list-units --type=service | grep <service-name>
```
**Bind mount a directory**
Make a directory available at another path, useful during chroot recovery or container setup:
```bash
mount --bind /dev /mnt/<newroot>/dev
```
**Chroot into another system**
Enter a mounted system's root as if it were the running OS. Useful for recovery, initramfs rebuilds, or bootloader fixes:
```bash
chroot /mnt/<newroot>
```
Typically preceded by binding the required pseudo-filesystems:
```bash
mount --bind /dev /mnt/<newroot>/dev
mount --bind /proc /mnt/<newroot>/proc
mount --bind /sys /mnt/<newroot>/sys
chroot /mnt/<newroot>
```
**Rebuild initramfs**
After kernel or driver changes, rebuild the initramfs and refresh the GRUB configuration:
```bash
update-initramfs -u
update-initramfs -u -k all
```
- `update-initramfs -u`: Rebuilds the initramfs for the currently running kernel.
- `-k all`: Rebuilds for all installed kernels.
**Rebuild initramfs for a specific kernel version:**
```bash
update-initramfs -c -k $(uname -r)
```
- `-c`: Create a new initramfs (instead of updating).
- `-k $(uname -r)`: Targets the currently running kernel version.
**Update GRUB:**
```bash
update-grub
```
- Scans for kernels and regenerates `/boot/grub/grub.cfg`.
### Change password of a tar/openssl archive
**Decrypt the archive**
@ -336,6 +418,60 @@ for file1 in $(find "$dir1" -type f); do
done
```
## NFS
**Show NFS exports from a server:**
```bash
showmount -e <hostname>
showmount -e localhost
```
**List active exports and their options on the server:**
```bash
exportfs -v
```
## Network Diagnostics
**Measure HTTP response timing:**
Breaks down the full request lifecycle — useful for diagnosing DNS, TLS, or TTFB issues:
```bash
curl -o /dev/null -s -w \
'Lookup: %{time_namelookup}s\nConnect: %{time_connect}s\nAppConnect: %{time_appconnect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n' \
https://<hostname>
```
- `time_namelookup`: DNS resolution time.
- `time_connect`: TCP connection time.
- `time_appconnect`: TLS handshake time.
- `time_starttransfer`: Time to first byte (TTFB).
- `-o /dev/null`: Discards the response body.
**High-frequency ping:**
Flood-style ping to stress-test latency or detect intermittent packet loss:
```bash
ping -i 0.002 <host>
```
- `-i 0.002`: Send a packet every 2ms. Requires root.
**Jumbo frame ping:**
Test whether the network path supports large MTU frames (useful for diagnosing MTU mismatches):
```bash
ping -s 1472 -i 0.01 <host>
```
- `-s 1472`: Payload size of 1472 bytes (1472 + 28-byte IP/ICMP header = 1500-byte MTU).
- Increase `-s` to test jumbo frames (e.g., `-s 8972` for 9000-byte MTU).
## Diagnosis
### Debian Upgrade Issues
@ -367,6 +503,28 @@ journalctl -b | grep -i "drm\|gpu\|display\|wayland\|monitor"
journalctl -b | grep -i "gnome-shell"
```
**Journal Filtering by Date and Keyword**
Search logs within a specific time window:
```bash
journalctl --since "<date>" --until "<date>" | grep -i <keyword>
```
Example:
```bash
journalctl --since "2026-01-01" --until "2026-01-02" | grep -i btrfs
```
**Kernel microcode events:**
```bash
journalctl -k | grep -i "microcode"
```
- `-k`: Show only kernel messages (equivalent to `dmesg` output via the journal).
## Fonts
**Download and Install Fonts**

324
notes/ssh.md
View File

@ -4,7 +4,9 @@
- [SSH](#ssh)
- [Table of Contents](#table-of-contents)
- [Placeholders](#placeholders)
- [Connect with specific key](#connect-with-specific-key)
- [Skip Host Key Verification](#skip-host-key-verification)
- [SSH Key Management](#ssh-key-management)
- [Verbose](#verbose)
- [Enable root login](#enable-root-login)
@ -16,93 +18,124 @@
- [Change SSH Port](#change-ssh-port)
- [Restart ssh](#restart-ssh)
## Placeholders
Replace the placeholders below with the appropriate values for your setup:
- **Connection**
- Username: `<username>` (e.g., john)
- Hostname: `<hostname>` (e.g., server.example.com)
- IP address: `<ip>` (e.g., 192.168.1.100)
- SSH port: `<port>` (e.g., 2222)
- SSH key: `<keyfile>` (e.g., ~/.ssh/id_rsa)
- Key comment: `<key-comment>` (e.g., john@example.com)
- Host alias: `<alias>` (e.g., myserver)
- **Paths**
- Local file: `<local-path>` (e.g., /home/user/file.txt)
- Remote path: `<remote-path>` (e.g., /home/user/file.txt)
- Local script: `<script-path>` (e.g., /home/user/script.sh)
- Project name: `<project>` (e.g., myapp)
## Connect with specific key
```bash
ssh -i /home/fabrice/.ssh/fabquenneville root@servername.fabq.ca
ssh -i /home/fabrice/.ssh/fabquenneville fabrice@servername.fabq.ca
ssh -i <keyfile> root@<hostname>
ssh -i <keyfile> <username>@<hostname>
```
## Skip Host Key Verification
Useful for ephemeral machines, VMs, or hosts that are frequently rebuilt where saved known_hosts entries would cause conflicts:
```bash
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null <username>@<hostname>
```
- `StrictHostKeyChecking=no`: Automatically accepts new or changed host keys without prompting.
- `UserKnownHostsFile=/dev/null`: Discards the host key entirely — nothing is saved to `~/.ssh/known_hosts`.
- ⚠️ Do not use on untrusted networks — this disables MITM protection.
## SSH Key Management
**Generate a new RSA SSH key pair with a 4096-bit key length**
```bash
ssh-keygen -t rsa -b 4096 -C "fabrice@fabq.ca" -f ~/.ssh/fabrice@fabq.ca
ssh-keygen -t rsa -b 4096 -C "<key-comment>" -f <keyfile>
```
- `ssh-keygen -t rsa -b 4096`: This command generates a new RSA SSH key pair with a key size of 4096 bits for enhanced security.
- `-C "fabrice@fabq.ca"`: This option adds a comment to the key, usually the email address of the key owner.
- `-f ~/.ssh/fabrice@fabq.ca`: This specifies the filename for the private key; the public key will be saved with the same name but with a `.pub` extension.
- `ssh-keygen -t rsa -b 4096`: Generates a new RSA SSH key pair with a key size of 4096 bits for enhanced security.
- `-C "<key-comment>"`: Adds a comment to the key, usually the email address of the key owner.
- `-f <keyfile>`: Specifies the filename for the private key; the public key will be saved with the same name but with a `.pub` extension.
**Copy the generated SSH keys to the remote server**
```bash
scp ~/.ssh/fabrice@fabq.ca* fabrice@servername.fabq.ca:~/.ssh/
scp <keyfile>* <username>@<hostname>:~/.ssh/
```
- `scp ~/.ssh/fabrice@fabq.ca*`: This command securely copies both the private and public keys to the remote server.
- `fabrice@servername.fabq.ca:~/.ssh/`: Specifies the destination path on the remote server where the keys will be copied.
- `scp <keyfile>*`: Securely copies both the private and public keys to the remote server.
- `<username>@<hostname>:~/.ssh/`: Specifies the destination path on the remote server where the keys will be copied.
**Install the public key on the remote server for passwordless authentication**
```bash
ssh-copy-id fabrice@192.168.1.100
ssh-copy-id fabrice@servername.fabq.ca
ssh-copy-id <username>@<ip>
ssh-copy-id <username>@<hostname>
```
- `ssh-copy-id "fabrice@servername.fabq.ca"`: This command installs the public key on the specified remote server, allowing for passwordless SSH login.
- `ssh-copy-id`: Installs the public key on the specified remote server, allowing for passwordless SSH login.
**Install the public key on multiple servers using specific private key**
**Install the public key on multiple servers using a specific private key**
```bash
ssh-copy-id -i /home/fabrice/.ssh/fabquenneville root@192.168.1.100
ssh-copy-id -i /home/fabrice/.ssh/fabquenneville fabrice@servername.fabq.ca
ssh-copy-id -i <keyfile> root@<ip>
ssh-copy-id -i <keyfile> <username>@<hostname>
```
- `ssh-copy-id -i /home/fabrice/.ssh/fabquenneville`: This specifies which private key to use for authentication when copying the public key.
- `root@192.168.1.100` and `fabrice@servername.fabq.ca`: These commands install the public key on the respective remote servers, allowing for secure, passwordless access.
- `-i <keyfile>`: Specifies which private key to use for authentication when copying the public key.
**Install the public key on the remote server for passwordless authentication manually**
This process is useful when ssh-copy-id is unavailable, or when you want more granular control over the manual setup of passwordless SSH authentication. The ssh-copy-id tool automatically installs your public key on the remote machine, but if you prefer or need to do it manually, these are the steps:
1. Create the .ssh directory if it doesn't exist and set proper permissions
This process is useful when `ssh-copy-id` is unavailable, or when you want more granular control over the setup. Follow these steps on the remote server:
```bash
mkdir -p /home/fabrice/.ssh
chmod 700 /home/fabrice/.ssh
```
1. Create the `.ssh` directory if it doesn't exist and set proper permissions:
2. Open the authorized_keys file in an editor and paste the public key (usually from ~/.ssh/id_rsa.pub on the local machine)
```bash
mkdir -p /home/<username>/.ssh
chmod 700 /home/<username>/.ssh
```
```bash
nano /home/fabrice/.ssh/authorized_keys
```
2. Open the `authorized_keys` file and paste the public key (usually from `~/.ssh/id_rsa.pub` on the local machine):
3. Set the correct permissions for the authorized_keys file
```bash
nano /home/<username>/.ssh/authorized_keys
```
```bash
chmod 600 /home/fabrice/.ssh/authorized_keys
```
3. Set the correct permissions for the `authorized_keys` file:
4. Ensure the ownership of the .ssh directory and its contents is set to the correct user
```bash
chmod 600 /home/<username>/.ssh/authorized_keys
```
```bash
chown -R fabrice:fabrice /home/fabrice/.ssh
```
4. Ensure the ownership of the `.ssh` directory and its contents is set to the correct user:
```bash
chown -R <username>:<username> /home/<username>/.ssh
```
## Verbose
- Use the 'ssh' command with the '-v' option to enable verbose mode, which provides detailed information about the connection process.
Use the `-v` option to enable verbose mode, which provides detailed information about the connection process:
```bash
ssh -i /home/fabrice/.ssh/fabquenneville -v root@servername.fabq.ca
ssh -i /home/fabrice/.ssh/fabquenneville -v fabrice@servername.fabq.ca
ssh -i <keyfile> -v root@<hostname>
ssh -i <keyfile> -v <username>@<hostname>
```
## Enable root login
- Modify the SSH configuration file to allow root login.
Modify the SSH configuration file to allow root login:
```bash
nano /etc/ssh/sshd_config
@ -123,284 +156,203 @@ PermitRootLogin yes
firewall-cmd --permanent --zone=public --add-service=ssh
```
- `firewall-cmd`: This is the command-line tool used to manage `firewalld`.
- `--permanent`: This option ensures that the change persists across reboots.
- `firewall-cmd`: The command-line tool used to manage `firewalld`.
- `--permanent`: Ensures that the change persists across reboots.
- `--zone=public`: Specifies the zone to which the rule applies. The "public" zone is typically used for untrusted networks.
- `--add-service=ssh`: This adds the SSH service to the specified zone, allowing incoming SSH connections.
- `--add-service=ssh`: Adds the SSH service to the specified zone, allowing incoming SSH connections.
**Examples of configuring other Linux firewalls**
1. **Using UFW (Uncomplicated Firewall)**
**Allow SSH traffic**
```bash
ufw allow ssh
```
- This command allows incoming SSH traffic through the firewall. UFW is designed to simplify the process of managing a firewall.
2. **Using iptables**
**Allow SSH traffic**
```bash
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
```
- `iptables`: This is a low-level tool for managing Linux firewalls.
- `-A INPUT`: Appends the rule to the INPUT chain.
- `-p tcp`: Specifies that this rule applies to TCP packets.
- `--dport 22`: Indicates that this rule applies to traffic on port 22 (the default SSH port).
- `-j ACCEPT`: Instructs the firewall to accept the specified traffic.
- `-p tcp --dport 22`: Matches TCP traffic on port 22.
- `-j ACCEPT`: Accepts the specified traffic.
3. **Using nftables**
**Allow SSH traffic**
```bash
nft add rule ip filter input tcp dport 22 accept
```
- `nft`: The command-line tool for interacting with the nftables framework.
- `add rule ip filter input`: Adds a new rule to the input chain of the filter table.
- `tcp dport 22`: Matches TCP packets directed to port 22.
- `accept`: Specifies that the matching packets should be accepted.
- `accept`: Accepts the matching packets.
**Note:** Be sure to reload or restart the firewall service after making changes to apply the new rules effectively.
**Note:** Reload or restart the firewall service after making changes to apply the new rules.
## SCP (Secure Copy Protocol)
- The `scp` command is used to securely transfer files and directories between local and remote systems over SSH.
The `scp` command securely transfers files and directories between local and remote systems over SSH.
**Copy Local File to Remote Server**
To copy a file from your local machine to a remote server, use the following syntax:
**Copy a local file to a remote server:**
```bash
scp /local/file/path fabrice@servername.fabq.ca:/remote/file/path
scp <local-path> <username>@<hostname>:<remote-path>
```
- `/local/file/path`: Specify the path to the local file you want to copy.
- `fabrice@servername.fabq.ca`: This is the user and remote server where the file will be copied.
- `/remote/file/path`: This is the destination path on the remote server.
**Copy a Directory**
To copy an entire directory, use the `-r` option, which stands for "recursive":
**Copy a directory recursively:**
```bash
scp -r /home/fabrice/foldername/ root@servername.fabq.ca:/remote/parent/
scp -r <local-path>/ <username>@<hostname>:<remote-path>/
```
- `-r`: This option enables recursive copying of directories and their contents.
- `/home/fabrice/foldername/`: The path to the local directory you wish to copy.
- `root@servername.fabq.ca:/remote/parent/`: The destination path on the remote server where the directory will be copied.
- `-r`: Enables recursive copying of directories and their contents.
**Copy a Configuration File Using a Specific SSH Key**
To copy a configuration file while specifying a particular SSH key for authentication, use the following command:
**Copy a file using a specific SSH key:**
```bash
scp -i /home/fabrice/.ssh/fabquenneville /mnt/workbench/webserver/projectname/config.ini fabrice@servername.fabq.ca:/mnt/workbench/projectname/
scp -i <keyfile> <local-path> <username>@<hostname>:<remote-path>
```
- `-i /home/fabrice/.ssh/fabquenneville`: This option specifies the identity file (private key) for authentication.
- `/mnt/workbench/webserver/projectname/config.ini`: The path to the local configuration file being transferred.
- `fabrice@servername.fabq.ca`: The user and server to which the file is being copied.
- `/mnt/workbench/projectname/`: The destination path on the remote server where the file will be stored.
- `-i <keyfile>`: Specifies the identity file (private key) for authentication.
## Send Remote Commands
**Execute Commands Directly on a Remote Server**
You can use the `ssh` command to execute various commands on a remote server. Here are some examples:
**Run a Local Script on a Remote Server**
**Run a local script on a remote server:**
```bash
ssh fabrice@servername.fabq.ca 'bash -s' < /local/path/to/script.sh
ssh <username>@<hostname> 'bash -s' < <script-path>
```
- This command will execute the local script located at `/local/path/to/script.sh` on the remote server.
**Remove a file:**
```bash
ssh fabrice@192.168.1.100 "rm /home/fabrice/filename.log"
ssh <username>@<ip> "rm <remote-path>"
```
**Mount all filesystems:**
```bash
ssh root@servername.fabq.ca "mount -a"
ssh root@<hostname> "mount -a"
```
**Reboot the remote server:**
```bash
ssh root@servername.fabq.ca "reboot -h now"
ssh root@<hostname> "reboot -h now"
```
**Connect using a host key alias:**
```bash
ssh -o "HostKeyAlias=servername" root@192.168.1.100
ssh -o "HostKeyAlias=<alias>" root@<ip>
```
## Test Connection, Add Alias, and Update Known Hosts
**Test Connection with Host Key Alias**
**Test connection with host key alias:**
Use the following commands to establish an SSH connection while specifying a host key alias. This helps avoid conflicts with existing entries in the `known_hosts` file.
Commands to establish an SSH connection while specifying a host key alias. This helps avoid conflicts with existing entries in the `known_hosts` file.
```bash
ssh -o 'HostKeyAlias=servername.fabq.ca' fabrice@192.168.1.100
ssh -o 'HostKeyAlias=servername' fabrice@192.168.1.100
ssh -o 'HostKeyAlias=<hostname>' <username>@<ip>
ssh -o 'HostKeyAlias=<alias>' <username>@<ip>
```
**Test Host Identity without Authenticating**
To test the identity of a remote server without fully authenticating and to check connectivity, use the following command:
**Test host identity without authenticating:**
```bash
ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=servername' fabrice@192.168.1.100 /bin/true
ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=<alias>' <username>@<ip> /bin/true
```
- `-e none`: Disables encryption for this command, which is useful in specific testing scenarios.
- `-o 'BatchMode=yes'`: Ensures that SSH does not prompt for user interaction, making it suitable for scripts.
- `/bin/true`: Executes a simple command that always returns success, confirming the connection without further actions.
- `-e none`: Disables escape character processing.
- `-o 'BatchMode=yes'`: Suppresses all prompts, suitable for scripts.
- `/bin/true`: Simple command that always returns success, confirming the connection without further actions.
This command allows you to verify that you can connect to the server while avoiding any authentication prompts.
**Retrieve public SSH keys from a remote server:**
**Retrieve Public SSH Keys**
To retrieve the public SSH keys from a remote server, use the following command:
```
ssh-keyscan -H servername.fabq.ca
```bash
ssh-keyscan -H <hostname>
```
- This command fetches the public SSH keys from the specified server, allowing you to add them to your `known_hosts` file.
- It does not establish a full SSH session and is primarily used for key retrieval, which helps ensure secure connections in future interactions.
- Fetches the server's public SSH keys without establishing a full session. Used to pre-populate `known_hosts`.
By using both commands, you can test connectivity to a remote server and gather its public SSH keys for secure authentication later.
**Add Alias to SSH Config for Easy Access**
To simplify your SSH connections, you can create an alias for your SSH connections by editing the `~/.ssh/config` file:
**Add an alias to SSH config for easy access:**
```ini
Host servername
HostName servername.fabq.ca
User fabrice
IdentityFile ~/.ssh/fabquenneville
Host <alias>
HostName <hostname>
User <username>
IdentityFile <keyfile>
```
- `Host servername`: This defines the alias you will use for the SSH connection.
- `HostName servername.fabq.ca`: This is the actual hostname of the remote server.
- `User fabrice`: This specifies the user to log in as.
- `IdentityFile ~/.ssh/fabquenneville`: This indicates the SSH key file to use for authentication.
## Remove Offending SSH Keys
When you encounter an "offending key" warning when connecting to a server, you can remove the old key from the `known_hosts` file. This is necessary if the server's host key has changed.
When a server's host key has changed, remove the old entry from `known_hosts`.
**View Known Hosts**
To view the contents of your `known_hosts` file, use:
**View known hosts:**
```bash
cat ~/.ssh/known_hosts
```
**Edit Known Hosts Manually (Optional)**
You can edit the `known_hosts` file manually if you prefer:
**Edit known hosts manually:**
```bash
nano ~/.ssh/known_hosts
```
**Update Known Hosts File with SSH Key**
**Update known hosts with current server key:**
```bash
ssh-keyscan -H servername.fabq.ca >> ~/.ssh/known_hosts
ssh-keyscan -H <hostname> >> ~/.ssh/known_hosts
```
- This command retrieves the public key of the specified server and appends it to your `known_hosts` file, allowing SSH to recognize the server during subsequent connections.
**Remove Offending Key by Hostname**
You can use the `ssh-keygen` command to remove specific keys from your `known_hosts` file. Here are examples for different scenarios:
- To remove the offending key for a specific server:
**Remove offending key by hostname or IP:**
```bash
ssh-keygen -R "servername.fabq.ca"
ssh-keygen -R "192.168.1.100"
ssh-keygen -R "<hostname>"
ssh-keygen -R "<ip>"
```
- To specify the `known_hosts` file directly:
**Remove offending key specifying the known_hosts file:**
```bash
ssh-keygen -f "/home/fabrice/.ssh/known_hosts" -R "servername.fabq.ca"
ssh-keygen -f "/root/.ssh/known_hosts" -R "192.168.1.100"
ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "servername.fabq.ca"
ssh-keygen -f "/home/<username>/.ssh/known_hosts" -R "<hostname>"
ssh-keygen -f "/root/.ssh/known_hosts" -R "<ip>"
ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "<hostname>"
```
**Summary of Key Removal**
You can also use a shorthand command to remove the offending key without specifying the file:
```bash
ssh-keygen -R servername.fabq.ca
```
This will automatically target the correct `known_hosts` file based on your user and system configuration.
## Change SSH Port
To enhance security, you may want to change the default SSH port (22) to a custom port. Follow these steps:
**1. Edit the SSH Configuration File**
Open the SSH daemon configuration file using a text editor:
**1. Edit the SSH configuration file:**
```bash
nano /etc/ssh/sshd_config
```
Edit the following line to set a new port (e.g., port 2222):
Set the desired port:
```ini
Port 2222
Port <port>
```
- Locate the line that specifies the port (usually `#Port 22`) and change it to your desired port number (e.g., `Port 2222`).
- Make sure to uncomment the line by removing the `#`.
**2. Create Directory for Systemd Override**
If you're using systemd, create a directory for the SSH socket override:
**2. Create directory for systemd override:**
```bash
mkdir -p /etc/systemd/system/ssh.socket.d
```
**3. Create an Override Configuration File**
Create or edit the override configuration file for the SSH socket:
**3. Create the override configuration file:**
```bash
nano /etc/systemd/system/ssh.socket.d/override.conf
```
- Add the following lines to specify the custom port:
```ini
[Socket]
ListenPort=2222 # Replace with your desired port number
ListenPort=<port>
```
**4. (Optional) Edit the Sockets Target Configuration**
@ -411,32 +363,20 @@ You may also want to edit the sockets target configuration to ensure it points t
nano /etc/systemd/system/sockets.target.wants/ssh.socket
```
- Make any necessary adjustments based on your custom port.
**5. Restart the SSH service to apply the changes**
After making changes, restart the SSH service to apply the new configuration:
**5. Restart the SSH service to apply the changes:**
```bash
systemctl restart sshd
```
**6. (Optional) Verify the New Port**
To verify that SSH is listening on the new port, you can use:
**6. Verify the new port:**
```bash
netstat -tuln | grep LISTEN
```
This will display the ports currently being listened to, allowing you to confirm that your changes were successful.
## Restart ssh
**Restart the SSH service to apply changes**
To restart the SSH service, use the following command:
```bash
systemctl restart sshd
```

216
notes/storage.md
View File

@ -11,10 +11,15 @@
- [Inspect fstab](#inspect-fstab)
- [Find Device Path from UUID](#find-device-path-from-uuid)
- [Power On Hours](#power-on-hours)
- [Swap](#swap)
- [Partitions and Filesystems](#partitions-and-filesystems)
- [TRIM](#trim)
- [Mounting](#mounting)
- [SMART Diagnostics](#smart-diagnostics)
- [Hardware Monitoring](#hardware-monitoring)
- [Kernel Messages](#kernel-messages)
- [Badblocks](#badblocks)
- [Hex Dump](#hex-dump)
- [Cloning drives and images with dd](#cloning-drives-and-images-with-dd)
- [Benchmarking](#benchmarking)
- [USB Devices](#usb-devices)
@ -57,6 +62,14 @@ lsblk -f
- `-f`: Add filesystem type, label, and UUID to the tree.
**Exclude loop devices from the listing**
```bash
lsblk -e 7
```
- `-e 7`: Excludes device major number 7 (loop devices), keeping the output clean on systems with many snaps or loop mounts.
**Disk space usage for all mounted filesystems**
```bash
@ -92,6 +105,7 @@ Using `blkid`:
```bash
blkid | grep <uuid>
blkid -U <uuid>
blkid <partition>
```
### Power On Hours
@ -100,6 +114,15 @@ Check power-on hours across multiple drives at once:
```bash
for dev in /dev/sd{a,b,c,d}; do echo -n "$dev: "; smartctl -a $dev | grep "Power_On_Hours"; done
for dev in /dev/sd{a..d}; do echo -n "$dev: "; smartctl -a $dev | grep "Power_On_Hours"; done
```
### Swap
Check Swap currently used by the system:
```bash
swapon --show
```
## Partitions and Filesystems
@ -116,6 +139,28 @@ fdisk -l <device>
fsck <partition>
```
## TRIM
TRIM allows the OS to inform the drive which blocks are no longer in use, maintaining performance on SSDs and NVMe drives over time.
**Run TRIM once manually across all mounted filesystems:**
```bash
fstrim -av
```
- `-a`: Run on all mounted filesystems that support TRIM.
- `-v`: Verbose — reports how much space was freed per filesystem.
**Enable the weekly TRIM timer:**
```bash
systemctl enable --now fstrim.timer
```
- Debian/Ubuntu run this weekly by default once enabled.
- Check timer status with `systemctl status fstrim.timer`.
## Mounting
**Validate all `fstab` entries**
@ -141,6 +186,12 @@ findmnt <device>
mount | grep <device>
```
**Check active mounts for a specific mountpoint:**
```bash
cat /proc/mounts | grep <mountpoint>
```
**Apply fstab changes without rebooting**
```bash
@ -168,6 +219,13 @@ smartctl -a <device>
smartctl -i <device>
```
```bash
for dev in /dev/sd[a-z] /dev/nvme[0-9]n[0-9]; do
echo "--- $dev ---"
smartctl -i $dev | grep -Ei "Model|Serial Number|Capacity"
done
```
**Run a short SMART test:**
```bash
@ -192,6 +250,133 @@ smartctl -a <device>
smartctl -A <device>
```
**Filter for key health attributes:**
Check the most important failure indicators in a single line:
```bash
smartctl -A <device> | grep -E "Reallocated|Pending|UDMA_CRC"
```
**Check multiple health attributes at once:**
```bash
smartctl -a <device> | grep -E "Power_On_Hours|Load_Cycle_Count|Reallocated_Sector_Ct"
```
**Check temperatures across all drives:**
Scans all SMART-capable devices and prints their temperature:
```bash
smartctl --scan | awk '{print $1}' | while read dev; do \
echo -n "$dev: "; \
smartctl -A $dev | grep -iE 'Temperature|Airflow_Temp' | awk '
/Temperature_Celsius/ {print $10 "°C"}
/Airflow_Temperature_Cel/ {print $10 "°C"}
/Temperature:/ {print $2 "°C"}
' | head -n 1; \
done
```
**Watch drive temperatures continuously:**
```bash
watch -n 5 "smartctl --scan | awk '{print \$1}' | while read dev; do \
echo -n \"\$dev: \"; \
smartctl -A \$dev | grep -iE 'Temperature|Airflow_Temp' | awk '\
/Temperature_Celsius/ {print \$10 \"°C\"} \
/Airflow_Temperature_Cel/ {print \$10 \"°C\"} \
/Temperature:/ {print \$2 \"°C\"}' | head -n 1; \
done"
```
## Hardware Monitoring
**Install lm-sensors:**
```bash
apt install lm-sensors
```
**Detect available sensor chips:**
Run once after installation to probe for hardware sensors:
```bash
sensors-detect
```
**Display current sensor readings:**
Shows CPU, GPU, and motherboard temperatures, fan speeds, and voltages:
```bash
sensors
```
## Kernel Messages
**Tail the most recent kernel messages:**
```bash
dmesg | tail -n 25
```
**Show only errors and warnings:**
```bash
dmesg --level=err,warn
```
**Show kernel messages with human-readable timestamps:**
```bash
dmesg -T
```
**Filter for NVMe events:**
```bash
dmesg | grep -i nvme
dmesg -w | grep -i nvme
```
- `-w`: Follow — print new messages as they arrive (like `tail -f`).
**Filter for ATA/SCSI/SATA/NVMe device events:**
```bash
dmesg | grep -i -E 'scsi|ata|nvme|sata'
```
**Filter for I/O errors:**
```bash
dmesg | grep -i "I/O error"
```
**Filter for ATA/SCSI/SATA/NVMe device errors:**
```bash
dmesg | grep -i -E 'scsi|ata|nvme|sata'
```
**Map ATA port number to block device name:**
When `dmesg` reports an error on e.g. `ata7` and you need to identify which physical drive that is:
```bash
ls -l /sys/class/block/ | grep ata<port-number>
dmesg -T | grep -iE "ata"
```
**Filter for BTRFS events:**
```bash
dmesg | grep -i btrfs
```
## Badblocks
**Read-only test:**
@ -221,6 +406,37 @@ Overwrites all data — use only on blank drives or drives to be deleted.
badblocks -wsv <device>
```
## Hex Dump
Inspect raw bytes on a device directly, useful for verifying partition tables, boot sectors, or investigating corruption.
**View the first 512 bytes (MBR / partition table):**
```bash
hexdump -C -n 512 <device>
```
**View 1 MB of data starting at a specific offset:**
```bash
hexdump -C -s 1G -n 1M <device>
```
- `-C`: Canonical format — hex on the left, ASCII on the right.
- `-n <length>`: Number of bytes to read.
- `-s <offset>`: Skip to this offset before reading.
**Extract readable strings from a raw device:**
Useful for locating file paths, filenames, or metadata remnants directly on a block device:
```bash
strings <device> | grep -C 200 "<search-term>" > <output-file>.txt
```
- `-C 200`: Show 200 lines of context around each match.
- Redirect to a file — output can be very large on multi-TB drives.
## Cloning drives and images with dd
**Clone a drive or create an image:**