fabrice/scripts-fabq
fabrice/scripts-fabq
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: split and clean drive/storage notes into three focused files

Browse Source 
* notes/btrfs.md : remove generic drive commands, genericize all UUIDs/paths/hostnames, add Placeholders section, add command shorthands table, collapse backup procedures into a single generic template
* notes/linux.md : remove all storage and drive-related content, genericize all hardcoded usernames/hostnames/paths
* notes/storage.md : new file consolidating all storage tooling (smartctl, badblocks, dd, hdparm, fsck, findmnt, fstab) from both btrfs.md and linux.md
This commit is contained in:
Fabrice Quenneville 2026-04-05 18:22:20 -04:00
parent d1767e7409
commit 13c0acd990
3 changed files with 629 additions and 546 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

501
notes/btrfs.md
View File

@ -4,377 +4,259 @@
- [BTRFS](#btrfs) - [BTRFS](#btrfs)
- [Table of Contents](#table-of-contents) - [Table of Contents](#table-of-contents)
- [Information on Drives](#information-on-drives) - [Placeholders](#placeholders)
- [BTRFS Command Shorthands](#btrfs-command-shorthands)
- [Information on Filesystem](#information-on-filesystem) - [Information on Filesystem](#information-on-filesystem)
- [Backup Procedures](#backup-procedures)
- [Recovery](#recovery)
- [Drive Manipulation](#drive-manipulation) - [Drive Manipulation](#drive-manipulation)
- [Replace Drives](#replace-drives) - [Replace Drives](#replace-drives)
- [Filesystem Manipulation](#filesystem-manipulation) - [Filesystem Manipulation](#filesystem-manipulation)
- [Upgrading Btrfs block group cache to V2](#upgrading-btrfs-block-group-cache-to-v2) - [Upgrading Btrfs Block Group Cache to V2](#upgrading-btrfs-block-group-cache-to-v2)
- [Defrag](#defrag)
- [Balances](#balances) - [Balances](#balances)
- [Scrub](#scrub) - [Scrub](#scrub)
- [Snapshots](#snapshots) - [Snapshots](#snapshots)
- [Create Snapshots](#create-snapshots)
- [Delete Snapshots](#delete-snapshots)
- [Backup Procedures](#backup-procedures)
- [Recovery](#recovery)
## Information on Drives ## Placeholders
**List of Drives and Mountpoints** Replace the placeholders below with the appropriate values for your setup:
To check all attached drives: - **Devices**
- Block device: `<device>` (e.g., /dev/sda)
- Source device to replace: `<source-device>` (e.g., /dev/sdb)
- Target device: `<target-device>` (e.g., /dev/sdc)
- UUID: `<uuid>` (e.g., a1b2c3d4-e5f6-7890-abcd-ef1234567890)
```bash - **Paths**
ls /dev/sd* - Mount point: `<mountpoint>` (e.g., /mnt/media)
ls /dev/nv* - Subvolume name: `<subvolume>` (e.g., root, home, backups, snapshots)
``` - Subvolume ID: `<subvolume-id>` (e.g., 257)
- Snapshot label: `<snapshot-label>` (e.g., 2024-09-15)
To view mountpoints and drive details such as names, sizes, and mountpoints: ## BTRFS Command Shorthands
```bash Most `btrfs` subcommands accept shortened aliases. The table below lists the common ones used throughout this document.
lsblk
df -h
cat /etc/fstab
```
**Drive Information** | Long form | Short form |
| ----------------------------- | ----------------------- |
To get detailed information and serial number of a specific drive: | `btrfs filesystem` | `btrfs fi` |
| `btrfs filesystem show` | `btrfs fi show` |
```bash | `btrfs filesystem usage` | `btrfs fi usage` |
smartctl -i /dev/sdc | `btrfs filesystem df` | `btrfs fi df` |
``` | `btrfs filesystem resize` | `btrfs fi resize` |
| `btrfs filesystem defragment` | `btrfs fi defrag` |
**Find the Device Path from UUID** | `btrfs subvolume` | `btrfs sub` |
| `btrfs subvolume list` | `btrfs sub list` |
Using lsblk: | `btrfs subvolume snapshot` | `btrfs sub snap` |
| `btrfs subvolume delete` | `btrfs sub del` |
```bash | `btrfs subvolume get-default` | `btrfs sub get-default` |
lsblk -o NAME,UUID,MOUNTPOINT | `btrfs subvolume set-default` | `btrfs sub set-default` |
``` | `btrfs device` | `btrfs dev` |
| `btrfs device add` | `btrfs dev add` |
Using blkid: | `btrfs device usage` | `btrfs dev usage` |
| `btrfs device stats` | `btrfs dev stats` |
```bash | `btrfs balance start` | `btrfs bal start` |
blkid | grep <UUID> | `btrfs balance status` | `btrfs bal status` |
``` | `btrfs balance cancel` | `btrfs bal cancel` |
| `btrfs inspect-internal` | `btrfs insp` |
## Information on Filesystem ## Information on Filesystem
**Show Basic Filesystem Information** **Show Basic Filesystem Information**
To display basic information (size, IDs, paths, etc.) for the specified mountpoint: Display basic information (size, IDs, paths, etc.) for the specified mountpoint:
```bash ```bash
btrfs fi show /mnt/media/ btrfs fi show <mountpoint>
``` ```
**Display Detailed Usage Information** **Display detailed usage information**
To show detailed usage information (allocated, unallocated, etc.) for the specified mountpoint: To show detailed usage information (allocated, unallocated, free, used, etc.) for the specified mountpoint:
```bash ```bash
btrfs fi usage /mnt/media btrfs fi usage <mountpoint>
``` ```
**Display detailed usage information as a table**
```bash
btrfs fi usage -T <mountpoint>
```
- `-T` The tabular flag gives you a nice grid that shows exactly how much Data, Metadata, and System space is allocated per device.
**Display Detailed Allocation Information** **Display Detailed Allocation Information**
To view detailed allocation information (block groups, used space) for the specified mountpoint: View block groups and used space:
```bash ```bash
btrfs fi df /mnt/media btrfs fi df <mountpoint>
``` ```
**Get Detailed Device Usage Statistics** **Get Detailed Device Usage Statistics**
To get detailed device usage statistics (physical size, unallocated space, RAID levels, etc.) for a BTRFS filesystem: Physical size, unallocated space, RAID levels, etc.:
```bash ```bash
btrfs device usage /mnt/media btrfs device usage <mountpoint>
``` ```
**Scan and Display BTRFS Information** **Scan and Display BTRFS Information**
To scan and display BTRFS information for all devices or a specific drive: Scan all devices or a specific drive:
```bash ```bash
btrfs device scan /dev/sda/ btrfs device scan <device>
``` ```
**Retrieve Statistics and Error Information** **Retrieve Statistics and Error Information**
To get statistics and error information (read errors, write errors, flush errors, etc.) for the specified mountpoint: Read errors, write errors, flush errors, etc.:
```bash ```bash
btrfs device stats /mnt/media btrfs device stats <mountpoint>
``` ```
**List BTRFS Subvolumes** **List BTRFS Subvolumes**
To list BTRFS subvolumes:
```bash ```bash
btrfs subvolume list / btrfs subvolume list <mountpoint>
btrfs subvolume list /home/fabrice
btrfs subvolume list /mnt/workbench
``` ```
**Default Subvolume** **Default Subvolume**
To check if a non-standard subvolume is set as the default: Check if a non-standard subvolume is set as the default:
```bash ```bash
btrfs subvol get-default /mnt/tmp/ btrfs subvol get-default <mountpoint>
btrfs subvol list /mnt/tmp/ btrfs subvol list <mountpoint>
``` ```
To change the default subvolume if a non-standard one is set: Change the default subvolume:
```bash ```bash
btrfs subvol set-default 257 /mnt/tmp/ btrfs subvol set-default <subvolume-id> <mountpoint>
``` ```
**Verify Current Cache Version** **Verify Current Cache Version**
To check if your filesystem is using cache V1 by device: Check if your filesystem is using cache V1 by device:
```bash ```bash
btrfs inspect-internal dump-super -f /dev/<device> | grep cache_generation btrfs inspect-internal dump-super -f <device> | grep cache_generation
``` ```
To check if your filesystem is using cache V1 by UUID: By UUID:
```bash ```bash
btrfs inspect-internal dump-super -f $(blkid -U <UUID>) | grep cache_generation btrfs inspect-internal dump-super -f $(blkid -U <uuid>) | grep cache_generation
``` ```
- If cache_generation is present, it indicates cache V1 is in use. If it's absent, the filesystem is already using V2. - If `cache_generation` is present, cache V1 is in use. If absent, the filesystem is already using V2.
## Backup Procedures
**Desktop Backup: Root and Home**
1. Mount snapshot location:
```bash
mount UUID=394decca-4780-47c9-9ae3-e4d03681a791 -o subvol=snapshots /mnt/snapshots
```
2. Create snapshots for root and home:
```bash
btrfs subvolume snapshot / "/mnt/snapshots/root/2021-05-23 - Fedora 34 upgrade"
btrfs subvolume snapshot /home "/mnt/snapshots/home/2021-05-23 - Fedora 34 upgrade"
```
3. Unmount after creating snapshots:
```bash
umount /mnt/snapshots
```
**Data Backup: Workbench, Documents, Education**
1. Mount snapshot location:
```bash
mount UUID=72e1770a-9fc0-461e-88d3-db640ff53dd9 -o subvol=snapshots /mnt/snapshots
```
2. Create snapshots for multiple directories:
```bash
btrfs subvolume snapshot /mnt/workbench "/mnt/snapshots/workbench/2021-05-23 - Fedora 34 upgrade"
btrfs subvolume snapshot /home/fabrice/Documents "/mnt/snapshots/Documents/2021-05-23 - Fedora 34 upgrade"
btrfs subvolume snapshot /home/fabrice/Education "/mnt/snapshots/Education/2021-05-23 - Fedora 34 upgrade"
```
3. Unmount after creating snapshots:
```bash
umount /mnt/snapshots
```
**STOR1 Fedora Backup**
1. Mount snapshot location:
```bash
mount UUID=e4fd608e-cfe8-4c10-b6d0-03b05bae8aa6 -o subvol=snapshots /mnt/snapshots
```
2. Create snapshot:
```bash
btrfs subvolume snapshot / "/mnt/snapshots/root/2021-06-06"
```
3. Unmount after creating snapshot:
```bash
umount /mnt/snapshots
```
**STOR1 Debian Backup**
1. Mount snapshot location:
```bash
mount UUID=c9a77f3c-626f-47bd-b4e3-9a094bea287f -o subvol=snapshots /mnt/snapshots
```
2. Create snapshot:
```bash
btrfs subvolume snapshot / "/mnt/snapshots/root/2021-07-12 - post mostly setup"
```
3. Unmount after creating snapshot:
```bash
umount /mnt/snapshots
```
**STOR2 Backup**
1. Mount snapshot location:
```bash
mount UUID=30bd5e0e-e781-4e87-8fb8-ea5606403b15 -o subvol=snapshots /mnt/snapshots
```
2. Create snapshot:
```bash
btrfs subvolume snapshot / "/mnt/snapshots/root/2021-06-06 - Fedora 34"
```
3. Unmount after creating snapshot:
```bash
umount /mnt/snapshots
```
## Recovery
**Mount a Subvolume with Recovery Options**
```bash
mount -o recovery,subvol=backups UUID=aa5c1d34-ecba-42a9-9339-8f7879d47536 /mnt/tmp
```
**Clear Cache During Mount**
```bash
mount -o clear_cache,subvol=backups UUID=aa5c1d34-ecba-42a9-9339-8f7879d47536 /mnt/tmp
```
**Data Restoration**
To restore data using `btrfs restore`:
```bash
btrfs restore -D /dev/sdb
```
## Drive Manipulation ## Drive Manipulation
**Mount Whole Drive** **Mount Whole Drive**
```bash ```bash
mount UUID=c9a77f3c-626f-47bd-b4e3-9a094bea287f /mnt/tmp mount UUID=<uuid> <mountpoint>
```
**Mount Subvolume by Name**
```bash
mount UUID=<uuid> -o subvol=<subvolume> <mountpoint>
``` ```
**Mount Subvolume by ID** **Mount Subvolume by ID**
```bash ```bash
btrfs subvol list / btrfs subvol list /
mount -o subvolid=5 /dev/disk/by-uuid/7a22514b-594a-43a3-8fdd-4df1530b5465 /mnt mount -o subvolid=<subvolume-id> /dev/disk/by-uuid/<uuid> <mountpoint>
``` ```
**Add a New Drive** **Add a New Drive**
To add a new drive to an existing BTRFS setup:
```bash ```bash
btrfs device add /dev/sdf /mnt/media/ btrfs device add <device> <mountpoint>
``` ```
**Resize Filesystem** **Resize Filesystem**
Grow the filesystem on a specific device to its maximum:
```bash ```bash
btrfs filesystem resize 1:max /mnt/media/ btrfs filesystem resize 1:max <mountpoint>
``` ```
**Create Subvolumes** **Create Subvolumes**
```bash ```bash
btrfs subvol create /mnt/tmp/root btrfs subvol create <mountpoint>/<subvolume>
btrfs subvol create /mnt/tmp/snapshots
``` ```
### Replace Drives ### Replace Drives
**Replace the source drive with the target drive:** **Start the replacement process:**
This command will start the replacement process where the data from the old drive (`/dev/sdb`) is copied over to the new drive (`/dev/sdj`). Copies data from the old drive to the new drive while the filesystem remains mounted:
```bash ```bash
btrfs replace start /dev/sdb /dev/sdj /mnt/media btrfs replace start <source-device> <target-device> <mountpoint>
``` ```
- `/dev/sdb`: Source drive to be replaced. - `<source-device>`: Drive to be replaced.
- `/dev/sdj`: Target drive to replace the source drive. - `<target-device>`: Drive to replace it with.
- `/mnt/media`: Mount point of the BTRFS filesystem. - `<mountpoint>`: Mount point of the BTRFS filesystem.
**Monitor the progress of the replacement:** **Monitor the progress:**
Once the replacement process has started, you can monitor its progress with the following command:
```bash ```bash
btrfs replace status /mnt/media btrfs replace status <mountpoint>
``` ```
- This will print the current status of the drive replacement operation, showing how much data has been migrated.
**Monitor progress interactively:** **Monitor progress interactively:**
For a more detailed, interactive status view of the replacement process, use the `-i` option:
```bash ```bash
btrfs replace status -i /mnt/media btrfs replace status -i <mountpoint>
``` ```
- `-i`: This flag provides an interactive mode where the progress is updated in real time. - `-i`: Updates progress in real time.
**Notes:** **Notes:**
- The `btrfs replace` command allows you to replace a faulty or underperforming drive without unmounting the filesystem, making it ideal for live systems. - `btrfs replace` works on a live mounted filesystem — no unmounting required.
- It can be used for upgrading storage by replacing smaller drives with larger ones, or for replacing failing drives. - Useful for both failing drive replacement and capacity upgrades.
- Ensure that the target drive has enough space to accommodate the data from the source drive. - Ensure the target drive has enough space to accommodate the source data.
## Filesystem Manipulation ## Filesystem Manipulation
### Upgrading Btrfs block group cache to V2 ### Upgrading Btrfs Block Group Cache to V2
**From a running system non-root filesystems** **Non-root filesystems (running system):**
```bash ```bash
mount -o remount,clear_cache,space_cache=v2 /mnt/<mount-point> mount -o remount,clear_cache,space_cache=v2 <mountpoint>
``` ```
**From a running system on root** **Root filesystem (running system):**
Check if your filesystem is using cache V1: 1. Check if using cache V1:
```bash ```bash
btrfs inspect-internal dump-super -f /dev/<device> | grep cache_generation btrfs inspect-internal dump-super -f <device> | grep cache_generation
``` ```
Enable Cache V2 2. Enable cache V2 via GRUB:
```bash ```bash
nano /etc/default/grub nano /etc/default/grub
# Locate the line starting with GRUB_CMDLINE_LINUX_DEFAULT or GRUB_CMDLINE_LINUX and add the following options: # Add to GRUB_CMDLINE_LINUX_DEFAULT or GRUB_CMDLINE_LINUX:
rootflags=clear_cache,space_cache=v2 # rootflags=clear_cache,space_cache=v2
``` ```
Example: Example:
@ -388,158 +270,209 @@ update-grub
reboot reboot
``` ```
Verify the Change 3. Verify the change:
```bash ```bash
btrfs inspect-internal dump-super -f /dev/<device> | grep cache_generation btrfs inspect-internal dump-super -f <device> | grep cache_generation
``` ```
Remove `clear_cache` Option 4. Remove `clear_cache` from GRUB after confirming:
```bash ```bash
nano /etc/default/grub nano /etc/default/grub
# Remove clear_cache from the rootflags. # Remove clear_cache from rootflags, then:
update-grub update-grub
``` ```
**From a live system** **From a live system:**
```bash ```bash
apt update apt update
apt install btrfs-progs apt install btrfs-progs
lsblk -o NAME,UUID lsblk -o NAME,UUID
blkid blkid
mount -o clear_cache,space_cache=v2 /dev/disk/by-uuid/<UUID> /mnt mount -o clear_cache,space_cache=v2 /dev/disk/by-uuid/<uuid> <mountpoint>
btrfs inspect-internal dump-super -f /dev/disk/by-uuid/<UUID> | grep cache_generation btrfs inspect-internal dump-super -f /dev/disk/by-uuid/<uuid> | grep cache_generation
umount /mnt umount <mountpoint>
``` ```
### Defrag
```bash
btrfs filesystem defrag -r -v -clzo <mountpoint>
```
- `-r`: Recursive.
- `-v`: Verbose.
- `-clzo`: Optional LZO compression to save space.
## Balances ## Balances
**Perform a Full Balance with Minimal Usage** **Full balance on nearly empty block groups:**
```bash ```bash
btrfs balance start --full-balance -dusage=0 -musage=0 /mnt/media/ btrfs balance start --full-balance -dusage=0 -musage=0 <mountpoint>
``` ```
- `--full-balance` is default but with a warning if not specified. - `--full-balance`: Default but with a warning if not specified.
- `-dusage=0` means only data block groups that are nearly empty (0% full) will be balanced. - `-dusage=0`: Only balance data block groups that are ~0% full.
- `-musage=0` means only metadata block groups that are nearly empty (0% full) will be balanced. - `-musage=0`: Only balance metadata block groups that are ~0% full.
**Perform a Full Balance on Partially Used Blocks** **Full balance on partially used block groups:**
```bash ```bash
btrfs balance start --full-balance -dusage=50 -musage=50 /mnt/media/ btrfs balance start --full-balance -dusage=50 -musage=50 <mountpoint>
``` ```
- `-dusage=50` means data block groups that are less than 50% full will be included in the balance process. - `-dusage=50`: Include data block groups less than 50% full.
- `-musage=50` means metadata block groups that are less than 50% full will also be balanced. - `-musage=50`: Include metadata block groups less than 50% full.
**Balance data in the background** **Balance data in the background:**
```bash ```bash
btrfs balance start --bg -d /mnt/media btrfs balance start --bg -d <mountpoint>
``` ```
**Balance metadata in the background** **Balance metadata in the background:**
```bash ```bash
btrfs balance start --bg -m /mnt/media btrfs balance start --bg -m <mountpoint>
``` ```
**Balance data and metadata in the background** **Balance data and metadata in the background:**
```bash ```bash
btrfs balance start --bg --full-balance -dusage=0 -musage=0 /mnt/media/ btrfs balance start --bg --full-balance -dusage=0 -musage=0 <mountpoint>
``` ```
**To balance 100 chunks of data** **Balance a limited number of chunks:**
```bash ```bash
btrfs balance start --bg -dlimit=100 /mnt/media/ btrfs balance start --bg -dlimit=100 <mountpoint>
``` ```
**Cancel Balance Operation** **Cancel a balance:**
```bash ```bash
btrfs balance cancel /mnt/media/ btrfs balance cancel <mountpoint>
``` ```
**Monitor Balance Status** **Monitor balance status:**
```bash ```bash
btrfs balance status /mnt/media/ btrfs balance status <mountpoint>
``` ```
## Scrub ## Scrub
**Start a Scrub Operation** **Start a scrub**
To start a scrub operation to verify data integrity: The scrub operation verifies data integrity against checksums
```bash ```bash
btrfs scrub start /mnt/media/ btrfs scrub start <mountpoint>
``` ```
**Check Scrub Status** **Check scrub status:**
To check the progress and status of the ongoing scrub:
```bash ```bash
btrfs scrub status /mnt/media/ btrfs scrub status <mountpoint>
``` ```
**Cancel a Scrub Operation** **Cancel a scrub:**
```bash ```bash
btrfs scrub cancel /mnt/media/ btrfs scrub cancel <mountpoint>
``` ```
## Snapshots ## Snapshots
**Create Snapshots** ### Create Snapshots
1. **Mount snapshot subvolume** 1. **Mount the snapshots subvolume:**
```bash ```bash
mount UUID=c9a77f3c-626f-47bd-b4e3-9a094bea287f -o subvol=snapshots /mnt/snapshots mount UUID=<uuid> -o subvol=snapshots <mountpoint>
``` ```
2. **Create a new snapshot** 2. **Create a snapshot:**
```bash ```bash
btrfs subvolume snapshot / "/mnt/snapshots/root/2021-06-26 - Debian install" btrfs subvolume snapshot <source-subvolume> "<mountpoint>/<snapshot-label>"
``` ```
3. **Unmount after creating snapshots** 3. **Unmount after creating:**
```bash ```bash
umount /mnt/snapshots umount <mountpoint>
``` ```
**Delete Snapshots** ### Delete Snapshots
1. **Mount subvolume containing snapshots** 1. **Mount the snapshots subvolume:**
```bash ```bash
mount -o subvol=snapshots /dev/disk/by-uuid/7a22514b-594a-43a3-8fdd-4df1530b5465 /mnt/snapshots/ mount -o subvol=snapshots /dev/disk/by-uuid/<uuid> <mountpoint>
``` ```
2. **List available snapshots** 2. **List available snapshots:**
```bash ```bash
btrfs subvol list /mnt/snapshots/ btrfs subvol list <mountpoint>
``` ```
3. **Delete the desired snapshot** 3. **Delete the desired snapshot:**
```bash ```bash
btrfs subvolume delete /mnt/snapshots/@rootfs/2024-09-15 btrfs subvolume delete <mountpoint>/<snapshot-label>
``` ```
4. **Unmount after deleting snapshots** 4. **Unmount after deleting:**
```bash ```bash
umount /mnt/snapshots/ umount <mountpoint>
```
## Backup Procedures
**Snapshot backup procedure:**
1. Mount snapshot location:
```bash
mount UUID=<uuid> -o subvol=snapshots <mountpoint>
```
2. Create snapshots for the desired subvolumes:
```bash
btrfs subvolume snapshot / "<mountpoint>/root/<snapshot-label>"
btrfs subvolume snapshot /home "<mountpoint>/home/<snapshot-label>"
btrfs subvolume snapshot <source-subvolume> "<mountpoint>/<subvolume>/<snapshot-label>"
```
3. Unmount after creating snapshots:
```bash
umount <mountpoint>
```
## Recovery
1. **Mount a Subvolume with Recovery Options:**
```bash
mount -o recovery,subvol=<subvolume> UUID=<uuid> <mountpoint>
```
2. **Clear Cache During Mount:**
```bash
mount -o clear_cache,subvol=<subvolume> UUID=<uuid> <mountpoint>
```
3. **Data Restoration with btrfs restore:**
```bash
btrfs restore -D <device>
``` ```

296
notes/linux.md
View File

@ -7,7 +7,6 @@
- [System Information](#system-information) - [System Information](#system-information)
- [Hardware Information](#hardware-information) - [Hardware Information](#hardware-information)
- [Software Information](#software-information) - [Software Information](#software-information)
- [Commands to Get Information About Linux Version, Kernel Version, and Release](#commands-to-get-information-about-linux-version-kernel-version-and-release)
- [User Management](#user-management) - [User Management](#user-management)
- [User Information](#user-information) - [User Information](#user-information)
- [Super User Management](#super-user-management) - [Super User Management](#super-user-management)
@ -15,8 +14,6 @@
- [System Management](#system-management) - [System Management](#system-management)
- [Change password of a tar/openssl archive](#change-password-of-a-taropenssl-archive) - [Change password of a tar/openssl archive](#change-password-of-a-taropenssl-archive)
- [Verify two possibly identical folders recursively](#verify-two-possibly-identical-folders-recursively) - [Verify two possibly identical folders recursively](#verify-two-possibly-identical-folders-recursively)
- [USB Devices](#usb-devices)
- [Test USB Key](#test-usb-key)
- [Diagnosis](#diagnosis) - [Diagnosis](#diagnosis)
- [Debian Upgrade Issues](#debian-upgrade-issues) - [Debian Upgrade Issues](#debian-upgrade-issues)
- [Wayland Issues](#wayland-issues) - [Wayland Issues](#wayland-issues)
@ -33,9 +30,6 @@ To gather detailed information about your hardware, use the following commands:
- **`hwinfo`**: Offers detailed information about hardware components and can be more verbose than `lshw`. - **`hwinfo`**: Offers detailed information about hardware components and can be more verbose than `lshw`.
- **`lsscsi`**: Lists SCSI devices, including disks and other SCSI-attached hardware. - **`lsscsi`**: Lists SCSI devices, including disks and other SCSI-attached hardware.
- **`lsusb`**: Shows information about USB devices connected to your system. - **`lsusb`**: Shows information about USB devices connected to your system.
- **`lsblk`**: Lists block devices such as hard drives and their partitions.
- **`df -H`**: Displays disk space usage in a human-readable format.
- **`fdisk -l`**: Lists all partitions on the system.
- **`dmidecode`**: Retrieves hardware information from the BIOS. Use: - **`dmidecode`**: Retrieves hardware information from the BIOS. Use:
- `dmidecode -t processor` for CPU details - `dmidecode -t processor` for CPU details
- `dmidecode -t memory` for RAM details - `dmidecode -t memory` for RAM details
@ -45,13 +39,11 @@ To gather detailed information about your hardware, use the following commands:
**Finding information on the Linux distribution** **Finding information on the Linux distribution**
# Commands to Get Information About Linux Version, Kernel Version, and Release
- **`lsb_release -a`**: Displays detailed information about the Linux distribution, including the distributor ID, description, release number, and codename. - **`lsb_release -a`**: Displays detailed information about the Linux distribution, including the distributor ID, description, release number, and codename.
- **`cat /etc/debian_version`**: Displays the version of the Debian distribution if you're running a Debian-based system (like Ubuntu). - **`cat /etc/debian_version`**: Displays the version of the Debian distribution if you're running a Debian-based system (like Ubuntu).
- **`cat /etc/os-release`**: Displays information about the operating system, such as the name, version, and ID of the distribution. - **`cat /etc/os-release`**: Displays information about the operating system, such as the name, version, and ID of the distribution.
- **`cat /etc/*release`**: Searches for any files in the `/etc/` directory that contain the word `release` and displays their contents. This typically includes more detailed distribution information. - **`cat /etc/*release`**: Searches for any files in the `/etc/` directory that contain the word `release` and displays their contents. This typically includes more detailed distribution information.
- **`cat /etc/*version`**: Similar to `cat /etc/*release`, but it looks for files containing the word `version`. It can provide additional version-related details. - **`cat /etc/*version`**: Similar to `cat /etc/*release`, but looks for files containing the word `version`. It can provide additional version-related details.
- **`hostnamectl`**: Displays system information related to the hostname and other metadata about the system. This may include the operating system, kernel version, and architecture. - **`hostnamectl`**: Displays system information related to the hostname and other metadata about the system. This may include the operating system, kernel version, and architecture.
**Finding Path to Binary** **Finding Path to Binary**
@ -59,17 +51,17 @@ To gather detailed information about your hardware, use the following commands:
To find the location of an executable binary, use: To find the location of an executable binary, use:
```bash ```bash
type composer type <binary-name>
``` ```
This command will show the path to the `composer` executable if it's available in your `PATH`. This command will show the path to a binary executable, ex `composer`, if it's available in your `PATH`.
**Number of Words in a File** **Number of Words in a File**
To count the number of words in a file, use: To count the number of words in a file, use:
```bash ```bash
wc filepath wc <filepath>
``` ```
This command will show the number of words along with other details like lines and characters. This command will show the number of words along with other details like lines and characters.
@ -79,11 +71,9 @@ This command will show the number of words along with other details like lines a
To count the number of lines in a file, use: To count the number of lines in a file, use:
```bash ```bash
wc -l filepath wc -l <filepath>
``` ```
This command will display the number of lines in the specified file.
## User Management ## User Management
### User Information ### User Information
@ -93,25 +83,23 @@ This command will display the number of lines in the specified file.
This variation of the adduser command uses the --gecos option to pre-fill the user's information (Full name, Room number, Work Phone, Home Phone, and Email) non-interactively, allowing you to automate user creation with predefined details. This variation of the adduser command uses the --gecos option to pre-fill the user's information (Full name, Room number, Work Phone, Home Phone, and Email) non-interactively, allowing you to automate user creation with predefined details.
```bash ```bash
adduser --gecos "Fabrice Quenneville,,,fabrice@fabq.ca" fabrice adduser --gecos "<full-name>,,,<email>" <username>
``` ```
This variation of the adduser command creates a system user named "aptly" with a Bash shell, no password login (--disabled-password), a specified home directory (/home/aptly), and adds the user to a new group, while using the --gecos option to set the full name as "Aptly repository". This variation creates a system user with a Bash shell, no password login (--disabled-password), a specified home directory (`/home/<username>`), and adds the user to a new group, while using the --gecos option to set the full name as `<service-description>`.
```bash ```bash
adduser --system --shell /bin/bash --gecos 'Aptly repository' --group --disabled-password --home /home/aptly aptly adduser --system --shell /bin/bash --gecos '<service-description>' --group --disabled-password --home /home/<username> <username>
``` ```
**List Users** **List Users**
To list all users from the `/etc/passwd` file, use: To list all users from the `/etc/passwd` file in alphabetical order, use:
```bash ```bash
awk -F':' '{ print $1}' /etc/passwd | sort awk -F':' '{ print $1}' /etc/passwd | sort
``` ```
This command extracts the usernames from the `/etc/passwd` file and sorts them in alphabetical order.
### Super User Management ### Super User Management
**Disable Root Login** **Disable Root Login**
@ -124,7 +112,7 @@ To disable root login via SSH, perform the following steps:
nano /etc/ssh/sshd_config nano /etc/ssh/sshd_config
``` ```
Comment out the line containing `PermitRootLogin` by adding a `#` at the beginning of the line. Comment out the line containing `PermitRootLogin`.
2. **Change Shell for Root User:** 2. **Change Shell for Root User:**
@ -132,34 +120,36 @@ To disable root login via SSH, perform the following steps:
nano /etc/passwd nano /etc/passwd
``` ```
Find the line starting with `root` and change `/bin/bash` to `/sbin/nologin` to disable login for the root user. Find the line starting with `root` and change `/bin/bash` to `/sbin/nologin`.
Save and close the file. Restart the SSH service for changes to take effect:
```bash ```bash
systemctl restart ssh systemctl restart ssh
``` ```
**Sudo Management**
**Add User to Sudo Group** **Add User to Sudo Group**
```bash ```bash
adduser fabrice sudo adduser <username> sudo
``` ```
**Update Sudoers File to Remove Password Requirement** **Update Sudoers File to Remove Password Requirement**
Edit the sudoers file: Edit the sudoers file with the default editor:
```bash ```bash
visudo visudo
``` ```
Edit the sudoers file with `nano`:
```bash
EDITOR=nano visudo
```
Add the following line to allow the user to execute commands without a password: Add the following line to allow the user to execute commands without a password:
```bash ```bash
fabrice ALL=(ALL) NOPASSWD:ALL <username> ALL=(ALL) NOPASSWD:ALL
``` ```
### Switch User ### Switch User
@ -167,82 +157,74 @@ fabrice ALL=(ALL) NOPASSWD:ALL
**Switch to Another User as Sudoer** **Switch to Another User as Sudoer**
```bash ```bash
sudo -i -u postgres sudo -i -u <username>
``` ```
This command switches to the `postgres` user with sudo privileges.
**Switch to Another User as Root** **Switch to Another User as Root**
```bash ```bash
su - postgres su - <username>
``` ```
This command switches to the `postgres` user with root privileges.
**Run command as specific user** **Run command as specific user**
```bash ```bash
sudo -u www-data somecommand and arguments sudo -u <username> <command>
``` ```
**Change shell of a user** **Change shell of a user**
```bash ```bash
chsh -s /bin/bash www-data chsh -s /bin/bash <username>
chsh -s /usr/sbin/nologin www-data chsh -s /usr/sbin/nologin <username>
``` ```
**Change user with specific shell** **Change user with specific shell**
```bash ```bash
sudo -u www-data bash sudo -u <username> bash
``` ```
## System Management ## System Management
**Ensure hostname or add alias** **Ensure hostname or add alias**
Set or update the hostname for your server.
```bash ```bash
nano /etc/hosts nano /etc/hosts
# Add the hostname alias: # Add the hostname alias:
# 127.0.1.1 local.servername.domain.com # 127.0.1.1 <hostname-intranet>
nano /etc/hostname nano /etc/hostname
# Set the main hostname: # Set the main hostname:
# 127.0.1.1 servername.domain.com servername # 127.0.1.1 <hostname-intranet> <hostname-short>
hostnamectl set-hostname servername.domain.com hostnamectl set-hostname <hostname-intranet>
``` ```
**Tar backup for a large number of small files** **Tar backup for a large number of small files**
These commands create backups using `tar` and transfer them securely over SSH.
Create a tar archive and transfer it to a remote server: Create a tar archive and transfer it to a remote server:
```bash ```bash
tar -c /path/to/dir | ssh fabrice@servername.domain.com 'tar -xvf - -C /absolute/path/to/remotedir' tar -c /path/to/dir | ssh <username>@<hostname-intranet> 'tar -xvf - -C /absolute/path/to/remotedir'
``` ```
Compress and transfer a folder, then store it as a .tar.gz file: Compress and transfer a folder, then store it as a `.tar.gz` file:
```bash ```bash
tar zcvf - /folder | ssh fabrice@servername.domain.com "cat > /backup/folder.tar.gz" tar zcvf - /folder | ssh <username>@<hostname-intranet> "cat > /backup/folder.tar.gz"
``` ```
Transfer a compressed .tar.gz file and extract it on the remote server: Transfer a compressed `.tar.gz` file and extract it on the remote server:
```bash ```bash
cat folder.tar.gz | ssh fabrice@servername.domain.com "tar zxvf -" cat folder.tar.gz | ssh <username>@<hostname-intranet> "tar zxvf -"
``` ```
Alternative method: change directory on the remote server before extracting: Alternative: change directory on the remote server before extracting:
```bash ```bash
cat folder.tar.gz | ssh fabrice@servername.domain.com "cd /path/to/dest/; tar zxvf -" cat folder.tar.gz | ssh <username>@<hostname-intranet> "cd /path/to/dest/; tar zxvf -"
``` ```
**List time zones** **List time zones**
@ -266,15 +248,13 @@ Alternatively, manually set the time zone by linking the correct file:
```bash ```bash
mv /etc/localtime /etc/localtime-old mv /etc/localtime /etc/localtime-old
ln -s /usr/share/zoneinfo/America/Toronto /etc/localtime ln -s /usr/share/zoneinfo/<timezone> /etc/localtime
``` ```
**Find a specific service** **Find a specific service**
Search for a specific service running on your system.
```bash ```bash
systemctl list-units --type=service | grep php systemctl list-units --type=service | grep <service-name>
``` ```
### Change password of a tar/openssl archive ### Change password of a tar/openssl archive
@ -283,66 +263,61 @@ systemctl list-units --type=service | grep php
To decrypt an `openssl`-encrypted archive using a password stored in a file: To decrypt an `openssl`-encrypted archive using a password stored in a file:
1. **Store your password in this file.** 1. **Store your password in a temporary file:**
```bash
nano $HOME/xyz001.txt
```
2. **Decrypt the archive**
Decrypt the archive using the password stored in xyz001.txt.
```bash ```bash
openssl aes-256-cbc -d -pbkdf2 -in servername-backup.tar.gz -out servername-backup.tar -pass file:$HOME/xyz001.txt nano $HOME/<filename>
``` ```
3. **Re-encrypt the archive with a new password** 2. **Decrypt the archive:**
```bash ```bash
nano $HOME/xyz001.txt openssl aes-256-cbc -d -pbkdf2 -in <archive>.tar.gz -out <archive>.tar -pass file:$HOME/<filename>
openssl aes-256-cbc -e -pbkdf2 -in servername-backup.tar -out servername-backup-new.tar.gz -pass file:$HOME/xyz001.txt ```
rm $HOME/xyz001.txt
3. **Re-encrypt the archive with a new password:**
```bash
nano $HOME/<filename>
openssl aes-256-cbc -e -pbkdf2 -in <archive>.tar -out <archive>-new.tar.gz -pass file:$HOME/<filename>
rm $HOME/<filename>
``` ```
**Decode / Extract** **Decode / Extract**
To decrypt and extract the contents of an encrypted archive directly into a directory: Decrypt and extract the contents directly into a directory:
```bash ```bash
nano $HOME/xyz001.txt nano $HOME/<filename>
openssl aes-256-cbc -d -pbkdf2 -in servername-backup.tar.gz -pass file:xyz001.txt | tar xz -C . openssl aes-256-cbc -d -pbkdf2 -in <archive>.tar.gz -pass file:<filename> | tar xz -C .
rm $HOME/xyz001.txt rm $HOME/<filename>
``` ```
### Verify two possibly identical folders recursively ### Verify two possibly identical folders recursively
**With `diff`** **With `diff`**
Check for differences between two directories, comparing all files recursively:
```bash ```bash
diff -r servername-files/data/servername-repositories/ servername-repositories/ diff -r <dir1>/ <dir2>/
``` ```
Outputs any differences found between the two directories.
**With `rsync`** **With `rsync`**
Use `rsync` to show differences without copying any data: Dry run — shows differences without copying any data:
```bash ```bash
rsync -avn servername-files/data/servername-repositories/ servername-repositories/ rsync -avn <dir1>/ <dir2>/
``` ```
- The `-n` flag means this is a dry run, which won’t make any changes. - `-n`: dry run, no changes made.
**With `cmp`** **With `cmp`**
This script compares files in two directories and identifies any differences between matching file names.
```bash ```bash
#!/bin/bash #!/bin/bash
dir1="servername-files/data/servername-repositories/" dir1="<dir1>/"
dir2="servername-repositories/" dir2="<dir2>/"
# Check if both directories exist before proceeding. # Check if both directories exist before proceeding.
if [ ! -d "$dir1" ] || [ ! -d "$dir2" ]; then if [ ! -d "$dir1" ] || [ ! -d "$dir2" ]; then
@ -361,149 +336,12 @@ for file1 in $(find "$dir1" -type f); do
done done
``` ```
## USB Devices
### Test USB Key
**Device Information**
Check if the system recognizes the device and show the latest system messages related to USB devices being connected:
```bash
lsusb
dmesg | tail -n 20
```
**Find Mount Points and Device Information**
Identify mount points, partitions, and other relevant details of mounted devices:
```bash
lsblk -f
df -h | grep /dev/sdc
findmnt /dev/sdc1
mount | grep /dev/sd
```
**Print Detailed Information About the USB Key**
View detailed partition and disk information:
```bash
fdisk -l /dev/sdc
```
**Test the File System**
Check and repair the filesystem on the USB key:
```bash
fsck /dev/sdc1
```
**Test Data Integrity**
Perform read/write tests to ensure the integrity of the USB key:
1. **Unmount the USB Key** (if mounted):
```bash
umount /media/fabrice/BD48-F8BB
```
2. **Write Test**:
```bash
dd if=/dev/zero of=/dev/sdc bs=4M count=256 status=progress
```
3. **Read Test**:
```bash
dd if=/dev/sdc of=/dev/null bs=4M count=256 status=progress
```
**Check for Bad Blocks**
Identify any bad sectors on the USB key:
- **Read-only test**:
```bash
badblocks -v /dev/sdc
```
- **Non-destructive read-write test**:
```bash
badblocks -nsv /dev/sdc
```
- The `-n` option performs a non-destructive read-write test.
- The `-s` option shows progress.
- The `-v` option is for verbose output.
**Perform a SMART Test**
Run SMART diagnostics to test the health of the USB key:
1. **Start a short SMART test**:
```bash
smartctl -t short /dev/sdc
```
2. **View test results**:
```bash
smartctl -a /dev/sdc
```
**Benchmark the Speed**
Measure the read speed of the USB key:
```bash
hdparm -t /dev/sdc
```
**Unmount and Safely Remove**
Unmount the USB key and safely remove it from the system:
```bash
umount /mnt/usb
eject /dev/sdc
```
**Switching two USB keys**
The following commands copy data between two USB drives, format one of them, and restore the data.
```bash
cp -r /media/fabrice/465A-759B "/tmp/Michael Allison"
umount /dev/sdc1
mkfs.vfat /dev/sdc1
umount /dev/sdc1
dd if=/dev/sdc of=/tmp/usb_image.img bs=4M status=progress
mkfs.vfat /dev/sdc1
cp -r "/tmp/Michael Allison" /media/fabrice/D67D-ADF8
umount /dev/sdc1
dd if=/tmp/usb_image.img of=/dev/sdc bs=4M status=progress
sync
```
## Diagnosis ## Diagnosis
### Debian Upgrade Issues ### Debian Upgrade Issues
**Apt Logs** **Apt Logs**
View the APT logs to check for package installation and updates history:
```bash ```bash
less /var/log/apt/history.log less /var/log/apt/history.log
``` ```
@ -533,19 +371,19 @@ journalctl -b | grep -i "gnome-shell"
**Download and Install Fonts** **Download and Install Fonts**
1. **Download the Font Archive**: 1. **Download the Font Archive:**
```bash ```bash
wget https://path/to/font/archive.tar.gz wget https://<font-archive-url>
``` ```
2. **Extract the Font Files**: 2. **Extract the Font Files:**
```bash ```bash
tar -xzvf font-archive.tar.gz tar -xzvf <font-archive>.tar.gz
``` ```
3. **Copy the Font Files to the Local Fonts Directory**: 3. **Copy the Font Files:**
**Local font directory** **Local font directory**
@ -567,14 +405,12 @@ journalctl -b | grep -i "gnome-shell"
**Update the Font Cache** **Update the Font Cache**
**Force a Reload of the Installed Font Cache**:
```bash ```bash
sudo su - sudo su -
fc-cache -fv fc-cache -fv
fc-cache -frv fc-cache -frv
``` ```
- **`-f`**: Force re-generation of apparently up-to-date cache files, overriding the timestamp checking. - **`-f`**: Force re-generation of cache files, overriding timestamp checking.
- **`-r`**: Erase all existing cache files and rescan. - **`-r`**: Erase all existing cache files and rescan.
- **`-v`**: Display status information while busy. - **`-v`**: Display status information while busy.

314
notes/storage.md Normal file
View File

@ -0,0 +1,314 @@
# Storage
## Table of Contents
- [Storage](#storage)
- [Table of Contents](#table-of-contents)
- [Placeholders](#placeholders)
- [Drive Information](#drive-information)
- [List Block Devices](#list-block-devices)
- [List Mountpoints and Usage](#list-mountpoints-and-usage)
- [Inspect fstab](#inspect-fstab)
- [Find Device Path from UUID](#find-device-path-from-uuid)
- [Power On Hours](#power-on-hours)
- [Partitions and Filesystems](#partitions-and-filesystems)
- [Mounting](#mounting)
- [SMART Diagnostics](#smart-diagnostics)
- [Badblocks](#badblocks)
- [Cloning drives and images with dd](#cloning-drives-and-images-with-dd)
- [Benchmarking](#benchmarking)
- [USB Devices](#usb-devices)
- [Device Recognition](#device-recognition)
- [Test Device Integrity](#test-device-integrity)
- [Switching Two USB Keys](#switching-two-usb-keys)
## Placeholders
Replace the placeholders below with the appropriate values for your setup:
- **Devices**
- Block device: `<device>` (e.g., /dev/sda)
- Partition: `<partition>` (e.g., /dev/sda1)
- UUID: `<uuid>` (e.g., a1b2c3d4-e5f6-7890-abcd-ef1234567890)
- **Paths**
- Mount point: `<mountpoint>` (e.g., /mnt/media)
- Directory path: `<dirpath>` (e.g., /mnt/data)
## Drive Information
### List Block Devices
Check all attached block devices by type:
```bash
ls /dev/sd*
ls /dev/nv*
```
### List Mountpoints and Usage
**Tree view of all block devices with sizes and mountpoints**
```bash
lsblk
lsblk -f
```
- `-f`: Add filesystem type, label, and UUID to the tree.
**Disk space usage for all mounted filesystems**
```bash
df -h
```
- `-h`: Human-readable.
**Disk space usage for a specific device**
```bash
df -h | grep <device>
```
### Inspect fstab
**View all configured mount entries**
```bash
cat /etc/fstab
```
### Find Device Path from UUID
Using `lsblk`:
```bash
lsblk -o NAME,UUID,MOUNTPOINT
```
Using `blkid`:
```bash
blkid | grep <uuid>
blkid -U <uuid>
```
### Power On Hours
Check power-on hours across multiple drives at once:
```bash
for dev in /dev/sd{a,b,c,d}; do echo -n "$dev: "; smartctl -a $dev | grep "Power_On_Hours"; done
```
## Partitions and Filesystems
**View partition table and disk details:**
```bash
fdisk -l <device>
```
**Check and repair a filesystem:**
```bash
fsck <partition>
```
## Mounting
**Validate all `fstab` entries**
```bash
findmnt --verify --verbose
```
**Dry-run mount of all fstab entries, reports what would succeed or fail**
```bash
mount -fav
```
- `-f`: Fake mount — goes through the motions without making the actual syscall.
- `-a`: Process all entries in `/etc/fstab` (excluding `noauto`).
- `-v`: Verbose output.
**Find mount point details:**
```bash
findmnt <device>
mount | grep <device>
```
**Apply fstab changes without rebooting**
```bash
mount -o remount -a
```
**Unmount and Safely Remove**
```bash
umount <device>
eject <device>
```
## SMART Diagnostics
**Get full SMART information for a drive:**
```bash
smartctl -a <device>
```
**Get drive identity and model info only:**
```bash
smartctl -i <device>
```
**Run a short SMART test:**
```bash
smartctl -t short <device>
```
**Run a long SMART test:**
```bash
smartctl -t long <device>
```
**View test results:**
```bash
smartctl -a <device>
```
**Check SMART device statistics (error counters, etc.):**
```bash
smartctl -A <device>
```
## Badblocks
**Read-only test:**
```bash
badblocks -v <device>
```
**Non-destructive read-write test:**
`Save to RAM -> Write Pattern -> Verify -> Restore`
```bash
badblocks -nsv <device>
```
- `-n`: Non-destructive read-write test.
- `-s`: Show progress.
- `-v`: Verbose output.
- Warning: If the computer loses power or the kernel panics after Step 2 but before Step 4, that specific block of data will be lost
**Destructive write test**
Overwrites all data — use only on blank drives or drives to be deleted.
```bash
badblocks -wsv <device>
```
## Cloning drives and images with dd
**Clone a drive or create an image:**
```bash
dd if=<source-device> of=<destination-device> bs=4M status=progress
dd if=<source-device> of=<image-file>.img bs=4M status=progress
```
**Restore from image:**
```bash
dd if=<image-file>.img of=<destination-device> bs=4M status=progress
sync
```
## Benchmarking
**Measure read speed with `hdparm`:**
```bash
hdparm -t <device>
```
## USB Devices
### Device Recognition
Check if the system recognizes the device and show recent USB-related kernel messages:
```bash
lsusb
dmesg | tail -n 20
```
### Test Device Integrity
Warning: The write test is destructive. All data on <device> will be permanently overwritten. Verify the target device with lsblk before running.
1. **Unmount the device** (if mounted):
```bash
umount <mountpoint>
```
2. **Write test:**
```bash
dd if=/dev/zero of=<device> bs=4M count=256 status=progress
```
3. **Read test:**
```bash
dd if=<device> of=/dev/null bs=4M count=256 status=progress
```
4. **Check for bad blocks:**
```bash
badblocks -wsv <device>
```
5. **Run a SMART test:**
```bash
smartctl -t short <device>
smartctl -a <device>
```
### Switching Two USB Keys
Copy data off, reformat, and restore:
```bash
cp -r /media/<username>/<volume-label> /tmp/<backup-folder>
umount <device-partition>
mkfs.vfat <device-partition>
cp -r /tmp/<backup-folder> /media/<username>/<new-volume-label>
umount <device-partition>
```
Or clone with `dd`:
```bash
dd if=<source-device> of=/tmp/usb_image.img bs=4M status=progress
mkfs.vfat <device-partition>
dd if=/tmp/usb_image.img of=<destination-device> bs=4M status=progress
sync
```