diff --git a/notes/btrfs.md b/notes/btrfs.md index 02b7da7..00d8da6 100644 --- a/notes/btrfs.md +++ b/notes/btrfs.md @@ -4,542 +4,475 @@ - [BTRFS](#btrfs) - [Table of Contents](#table-of-contents) - - [Information on Drives](#information-on-drives) + - [Placeholders](#placeholders) + - [BTRFS Command Shorthands](#btrfs-command-shorthands) - [Information on Filesystem](#information-on-filesystem) - - [Backup Procedures](#backup-procedures) - - [Recovery](#recovery) - [Drive Manipulation](#drive-manipulation) - [Replace Drives](#replace-drives) - [Filesystem Manipulation](#filesystem-manipulation) - - [Upgrading Btrfs block group cache to V2](#upgrading-btrfs-block-group-cache-to-v2) + - [Upgrading Btrfs Block Group Cache to V2](#upgrading-btrfs-block-group-cache-to-v2) + - [Defrag](#defrag) - [Balances](#balances) - [Scrub](#scrub) - [Snapshots](#snapshots) + - [Create Snapshots](#create-snapshots) + - [Delete Snapshots](#delete-snapshots) + - [Backup Procedures](#backup-procedures) + - [Recovery](#recovery) -## Information on Drives +## Placeholders -**List of Drives and Mountpoints** +Replace the placeholders below with the appropriate values for your setup: -To check all attached drives: +- **Devices** + - Block device: `` (e.g., /dev/sda) + - Source device to replace: `` (e.g., /dev/sdb) + - Target device: `` (e.g., /dev/sdc) + - UUID: `` (e.g., a1b2c3d4-e5f6-7890-abcd-ef1234567890) -```bash -ls /dev/sd* -ls /dev/nv* -``` +- **Paths** + - Mount point: `` (e.g., /mnt/media) + - Subvolume name: `` (e.g., root, home, backups, snapshots) + - Subvolume ID: `` (e.g., 257) + - Snapshot label: `` (e.g., 2024-09-15) -To view mountpoints and drive details such as names, sizes, and mountpoints: +## BTRFS Command Shorthands -```bash -lsblk -df -h -cat /etc/fstab -``` +Most `btrfs` subcommands accept shortened aliases. The table below lists the common ones used throughout this document. -**Drive Information** - -To get detailed information and serial number of a specific drive: - -```bash -smartctl -i /dev/sdc -``` - -**Find the Device Path from UUID** - -Using lsblk: - -```bash -lsblk -o NAME,UUID,MOUNTPOINT -``` - -Using blkid: - -```bash -blkid | grep -``` +| Long form | Short form | +| ----------------------------- | ----------------------- | +| `btrfs filesystem` | `btrfs fi` | +| `btrfs filesystem show` | `btrfs fi show` | +| `btrfs filesystem usage` | `btrfs fi usage` | +| `btrfs filesystem df` | `btrfs fi df` | +| `btrfs filesystem resize` | `btrfs fi resize` | +| `btrfs filesystem defragment` | `btrfs fi defrag` | +| `btrfs subvolume` | `btrfs sub` | +| `btrfs subvolume list` | `btrfs sub list` | +| `btrfs subvolume snapshot` | `btrfs sub snap` | +| `btrfs subvolume delete` | `btrfs sub del` | +| `btrfs subvolume get-default` | `btrfs sub get-default` | +| `btrfs subvolume set-default` | `btrfs sub set-default` | +| `btrfs device` | `btrfs dev` | +| `btrfs device add` | `btrfs dev add` | +| `btrfs device usage` | `btrfs dev usage` | +| `btrfs device stats` | `btrfs dev stats` | +| `btrfs balance start` | `btrfs bal start` | +| `btrfs balance status` | `btrfs bal status` | +| `btrfs balance cancel` | `btrfs bal cancel` | +| `btrfs inspect-internal` | `btrfs insp` | ## Information on Filesystem **Show Basic Filesystem Information** -To display basic information (size, IDs, paths, etc.) for the specified mountpoint: +Display basic information (size, IDs, paths, etc.) for the specified mountpoint: ```bash -btrfs fi show /mnt/media/ +btrfs fi show ``` -**Display Detailed Usage Information** +**Display detailed usage information** -To show detailed usage information (allocated, unallocated, etc.) for the specified mountpoint: +To show detailed usage information (allocated, unallocated, free, used, etc.) for the specified mountpoint: ```bash -btrfs fi usage /mnt/media +btrfs fi usage ``` +**Display detailed usage information as a table** + +```bash +btrfs fi usage -T +``` + +- `-T` The tabular flag gives you a nice grid that shows exactly how much Data, Metadata, and System space is allocated per device. + **Display Detailed Allocation Information** -To view detailed allocation information (block groups, used space) for the specified mountpoint: +View block groups and used space: ```bash -btrfs fi df /mnt/media +btrfs fi df ``` **Get Detailed Device Usage Statistics** -To get detailed device usage statistics (physical size, unallocated space, RAID levels, etc.) for a BTRFS filesystem: +Physical size, unallocated space, RAID levels, etc.: ```bash -btrfs device usage /mnt/media +btrfs device usage ``` **Scan and Display BTRFS Information** -To scan and display BTRFS information for all devices or a specific drive: +Scan all devices or a specific drive: ```bash -btrfs device scan /dev/sda/ +btrfs device scan ``` **Retrieve Statistics and Error Information** -To get statistics and error information (read errors, write errors, flush errors, etc.) for the specified mountpoint: +Read errors, write errors, flush errors, etc.: ```bash -btrfs device stats /mnt/media +btrfs device stats ``` **List BTRFS Subvolumes** -To list BTRFS subvolumes: - ```bash -btrfs subvolume list / -btrfs subvolume list /home/fabrice -btrfs subvolume list /mnt/workbench +btrfs subvolume list ``` **Default Subvolume** -To check if a non-standard subvolume is set as the default: +Check if a non-standard subvolume is set as the default: ```bash -btrfs subvol get-default /mnt/tmp/ -btrfs subvol list /mnt/tmp/ +btrfs subvol get-default +btrfs subvol list ``` -To change the default subvolume if a non-standard one is set: +Change the default subvolume: ```bash -btrfs subvol set-default 257 /mnt/tmp/ +btrfs subvol set-default ``` **Verify Current Cache Version** -To check if your filesystem is using cache V1 by device: +Check if your filesystem is using cache V1 by device: ```bash -btrfs inspect-internal dump-super -f /dev/ | grep cache_generation +btrfs inspect-internal dump-super -f | grep cache_generation ``` -To check if your filesystem is using cache V1 by UUID: +By UUID: ```bash -btrfs inspect-internal dump-super -f $(blkid -U ) | grep cache_generation +btrfs inspect-internal dump-super -f $(blkid -U ) | grep cache_generation ``` -- If cache_generation is present, it indicates cache V1 is in use. If it's absent, the filesystem is already using V2. - -## Backup Procedures - -**Desktop Backup: Root and Home** - -1. Mount snapshot location: - - ```bash - mount UUID=394decca-4780-47c9-9ae3-e4d03681a791 -o subvol=snapshots /mnt/snapshots - ``` - -2. Create snapshots for root and home: - - ```bash - btrfs subvolume snapshot / "/mnt/snapshots/root/2021-05-23 - Fedora 34 upgrade" - btrfs subvolume snapshot /home "/mnt/snapshots/home/2021-05-23 - Fedora 34 upgrade" - ``` - -3. Unmount after creating snapshots: - - ```bash - umount /mnt/snapshots - ``` - -**Data Backup: Workbench, Documents, Education** - -1. Mount snapshot location: - - ```bash - mount UUID=72e1770a-9fc0-461e-88d3-db640ff53dd9 -o subvol=snapshots /mnt/snapshots - ``` - -2. Create snapshots for multiple directories: - - ```bash - btrfs subvolume snapshot /mnt/workbench "/mnt/snapshots/workbench/2021-05-23 - Fedora 34 upgrade" - btrfs subvolume snapshot /home/fabrice/Documents "/mnt/snapshots/Documents/2021-05-23 - Fedora 34 upgrade" - btrfs subvolume snapshot /home/fabrice/Education "/mnt/snapshots/Education/2021-05-23 - Fedora 34 upgrade" - ``` - -3. Unmount after creating snapshots: - - ```bash - umount /mnt/snapshots - ``` - -**STOR1 Fedora Backup** - -1. Mount snapshot location: - - ```bash - mount UUID=e4fd608e-cfe8-4c10-b6d0-03b05bae8aa6 -o subvol=snapshots /mnt/snapshots - ``` - -2. Create snapshot: - - ```bash - btrfs subvolume snapshot / "/mnt/snapshots/root/2021-06-06" - ``` - -3. Unmount after creating snapshot: - - ```bash - umount /mnt/snapshots - ``` - -**STOR1 Debian Backup** - -1. Mount snapshot location: - - ```bash - mount UUID=c9a77f3c-626f-47bd-b4e3-9a094bea287f -o subvol=snapshots /mnt/snapshots - ``` - -2. Create snapshot: - - ```bash - btrfs subvolume snapshot / "/mnt/snapshots/root/2021-07-12 - post mostly setup" - ``` - -3. Unmount after creating snapshot: - - ```bash - umount /mnt/snapshots - ``` - -**STOR2 Backup** - -1. Mount snapshot location: - - ```bash - mount UUID=30bd5e0e-e781-4e87-8fb8-ea5606403b15 -o subvol=snapshots /mnt/snapshots - ``` - -2. Create snapshot: - - ```bash - btrfs subvolume snapshot / "/mnt/snapshots/root/2021-06-06 - Fedora 34" - ``` - -3. Unmount after creating snapshot: - - ```bash - umount /mnt/snapshots - ``` - -## Recovery - -**Mount a Subvolume with Recovery Options** - -```bash -mount -o recovery,subvol=backups UUID=aa5c1d34-ecba-42a9-9339-8f7879d47536 /mnt/tmp -``` - -**Clear Cache During Mount** - -```bash -mount -o clear_cache,subvol=backups UUID=aa5c1d34-ecba-42a9-9339-8f7879d47536 /mnt/tmp -``` - -**Data Restoration** - -To restore data using `btrfs restore`: - -```bash -btrfs restore -D /dev/sdb -``` +- If `cache_generation` is present, cache V1 is in use. If absent, the filesystem is already using V2. ## Drive Manipulation **Mount Whole Drive** ```bash -mount UUID=c9a77f3c-626f-47bd-b4e3-9a094bea287f /mnt/tmp +mount UUID= +``` + +**Mount Subvolume by Name** + +```bash +mount UUID= -o subvol= ``` **Mount Subvolume by ID** ```bash btrfs subvol list / -mount -o subvolid=5 /dev/disk/by-uuid/7a22514b-594a-43a3-8fdd-4df1530b5465 /mnt +mount -o subvolid= /dev/disk/by-uuid/ ``` **Add a New Drive** -To add a new drive to an existing BTRFS setup: - ```bash -btrfs device add /dev/sdf /mnt/media/ +btrfs device add ``` **Resize Filesystem** +Grow the filesystem on a specific device to its maximum: + ```bash -btrfs filesystem resize 1:max /mnt/media/ +btrfs filesystem resize 1:max ``` **Create Subvolumes** ```bash -btrfs subvol create /mnt/tmp/root -btrfs subvol create /mnt/tmp/snapshots +btrfs subvol create / ``` ### Replace Drives -**Replace the source drive with the target drive:** +**Start the replacement process:** -This command will start the replacement process where the data from the old drive (`/dev/sdb`) is copied over to the new drive (`/dev/sdj`). +Copies data from the old drive to the new drive while the filesystem remains mounted: ```bash -btrfs replace start /dev/sdb /dev/sdj /mnt/media +btrfs replace start ``` -- `/dev/sdb`: Source drive to be replaced. -- `/dev/sdj`: Target drive to replace the source drive. -- `/mnt/media`: Mount point of the BTRFS filesystem. +- ``: Drive to be replaced. +- ``: Drive to replace it with. +- ``: Mount point of the BTRFS filesystem. -**Monitor the progress of the replacement:** - -Once the replacement process has started, you can monitor its progress with the following command: +**Monitor the progress:** ```bash -btrfs replace status /mnt/media +btrfs replace status ``` -- This will print the current status of the drive replacement operation, showing how much data has been migrated. - **Monitor progress interactively:** -For a more detailed, interactive status view of the replacement process, use the `-i` option: - ```bash -btrfs replace status -i /mnt/media +btrfs replace status -i ``` -- `-i`: This flag provides an interactive mode where the progress is updated in real time. +- `-i`: Updates progress in real time. **Notes:** -- The `btrfs replace` command allows you to replace a faulty or underperforming drive without unmounting the filesystem, making it ideal for live systems. -- It can be used for upgrading storage by replacing smaller drives with larger ones, or for replacing failing drives. -- Ensure that the target drive has enough space to accommodate the data from the source drive. +- `btrfs replace` works on a live mounted filesystem — no unmounting required. +- Useful for both failing drive replacement and capacity upgrades. +- Ensure the target drive has enough space to accommodate the source data. ## Filesystem Manipulation -### Upgrading Btrfs block group cache to V2 +### Upgrading Btrfs Block Group Cache to V2 -**From a running system non-root filesystems** +**Non-root filesystems (running system):** ```bash -mount -o remount,clear_cache,space_cache=v2 /mnt/ +mount -o remount,clear_cache,space_cache=v2 ``` -**From a running system on root** +**Root filesystem (running system):** -Check if your filesystem is using cache V1: +1. Check if using cache V1: -```bash -btrfs inspect-internal dump-super -f /dev/ | grep cache_generation -``` + ```bash + btrfs inspect-internal dump-super -f | grep cache_generation + ``` -Enable Cache V2 +2. Enable cache V2 via GRUB: -```bash -nano /etc/default/grub -# Locate the line starting with GRUB_CMDLINE_LINUX_DEFAULT or GRUB_CMDLINE_LINUX and add the following options: -rootflags=clear_cache,space_cache=v2 -``` + ```bash + nano /etc/default/grub + # Add to GRUB_CMDLINE_LINUX_DEFAULT or GRUB_CMDLINE_LINUX: + # rootflags=clear_cache,space_cache=v2 + ``` -Example: + Example: -```bash -GRUB_CMDLINE_LINUX_DEFAULT="quiet splash rootflags=clear_cache,space_cache=v2" -``` + ```bash + GRUB_CMDLINE_LINUX_DEFAULT="quiet splash rootflags=clear_cache,space_cache=v2" + ``` -```bash -update-grub -reboot -``` + ```bash + update-grub + reboot + ``` -Verify the Change +3. Verify the change: -```bash -btrfs inspect-internal dump-super -f /dev/ | grep cache_generation -``` + ```bash + btrfs inspect-internal dump-super -f | grep cache_generation + ``` -Remove `clear_cache` Option +4. Remove `clear_cache` from GRUB after confirming: -```bash -nano /etc/default/grub -# Remove clear_cache from the rootflags. -update-grub -``` + ```bash + nano /etc/default/grub + # Remove clear_cache from rootflags, then: + update-grub + ``` -**From a live system** +**From a live system:** ```bash apt update apt install btrfs-progs lsblk -o NAME,UUID blkid -mount -o clear_cache,space_cache=v2 /dev/disk/by-uuid/ /mnt -btrfs inspect-internal dump-super -f /dev/disk/by-uuid/ | grep cache_generation -umount /mnt - +mount -o clear_cache,space_cache=v2 /dev/disk/by-uuid/ +btrfs inspect-internal dump-super -f /dev/disk/by-uuid/ | grep cache_generation +umount ``` +### Defrag + +```bash +btrfs filesystem defrag -r -v -clzo +``` + +- `-r`: Recursive. +- `-v`: Verbose. +- `-clzo`: Optional LZO compression to save space. + ## Balances -**Perform a Full Balance with Minimal Usage** +**Full balance on nearly empty block groups:** ```bash -btrfs balance start --full-balance -dusage=0 -musage=0 /mnt/media/ +btrfs balance start --full-balance -dusage=0 -musage=0 ``` -- `--full-balance` is default but with a warning if not specified. -- `-dusage=0` means only data block groups that are nearly empty (0% full) will be balanced. -- `-musage=0` means only metadata block groups that are nearly empty (0% full) will be balanced. +- `--full-balance`: Default but with a warning if not specified. +- `-dusage=0`: Only balance data block groups that are ~0% full. +- `-musage=0`: Only balance metadata block groups that are ~0% full. -**Perform a Full Balance on Partially Used Blocks** +**Full balance on partially used block groups:** ```bash -btrfs balance start --full-balance -dusage=50 -musage=50 /mnt/media/ +btrfs balance start --full-balance -dusage=50 -musage=50 ``` -- `-dusage=50` means data block groups that are less than 50% full will be included in the balance process. -- `-musage=50` means metadata block groups that are less than 50% full will also be balanced. +- `-dusage=50`: Include data block groups less than 50% full. +- `-musage=50`: Include metadata block groups less than 50% full. -**Balance data in the background** +**Balance data in the background:** ```bash -btrfs balance start --bg -d /mnt/media +btrfs balance start --bg -d ``` -**Balance metadata in the background** +**Balance metadata in the background:** ```bash -btrfs balance start --bg -m /mnt/media +btrfs balance start --bg -m ``` -**Balance data and metadata in the background** +**Balance data and metadata in the background:** ```bash -btrfs balance start --bg --full-balance -dusage=0 -musage=0 /mnt/media/ +btrfs balance start --bg --full-balance -dusage=0 -musage=0 ``` -**To balance 100 chunks of data** +**Balance a limited number of chunks:** ```bash -btrfs balance start --bg -dlimit=100 /mnt/media/ +btrfs balance start --bg -dlimit=100 ``` -**Cancel Balance Operation** +**Cancel a balance:** ```bash -btrfs balance cancel /mnt/media/ +btrfs balance cancel ``` -**Monitor Balance Status** +**Monitor balance status:** ```bash -btrfs balance status /mnt/media/ +btrfs balance status ``` ## Scrub -**Start a Scrub Operation** +**Start a scrub** -To start a scrub operation to verify data integrity: +The scrub operation verifies data integrity against checksums ```bash -btrfs scrub start /mnt/media/ +btrfs scrub start ``` -**Check Scrub Status** - -To check the progress and status of the ongoing scrub: +**Check scrub status:** ```bash -btrfs scrub status /mnt/media/ +btrfs scrub status ``` -**Cancel a Scrub Operation** +**Cancel a scrub:** ```bash -btrfs scrub cancel /mnt/media/ +btrfs scrub cancel ``` ## Snapshots -**Create Snapshots** +### Create Snapshots -1. **Mount snapshot subvolume** - -```bash -mount UUID=c9a77f3c-626f-47bd-b4e3-9a094bea287f -o subvol=snapshots /mnt/snapshots -``` - -2. **Create a new snapshot** - -```bash -btrfs subvolume snapshot / "/mnt/snapshots/root/2021-06-26 - Debian install" -``` - -3. **Unmount after creating snapshots** - -```bash -umount /mnt/snapshots -``` - -**Delete Snapshots** - -1. **Mount subvolume containing snapshots** +1. **Mount the snapshots subvolume:** ```bash - mount -o subvol=snapshots /dev/disk/by-uuid/7a22514b-594a-43a3-8fdd-4df1530b5465 /mnt/snapshots/ + mount UUID= -o subvol=snapshots ``` -2. **List available snapshots** +2. **Create a snapshot:** ```bash - btrfs subvol list /mnt/snapshots/ + btrfs subvolume snapshot "/" ``` -3. **Delete the desired snapshot** +3. **Unmount after creating:** ```bash - btrfs subvolume delete /mnt/snapshots/@rootfs/2024-09-15 + umount ``` -4. **Unmount after deleting snapshots** +### Delete Snapshots + +1. **Mount the snapshots subvolume:** ```bash - umount /mnt/snapshots/ + mount -o subvol=snapshots /dev/disk/by-uuid/ + ``` + +2. **List available snapshots:** + + ```bash + btrfs subvol list + ``` + +3. **Delete the desired snapshot:** + + ```bash + btrfs subvolume delete / + ``` + +4. **Unmount after deleting:** + + ```bash + umount + ``` + +## Backup Procedures + +**Snapshot backup procedure:** + +1. Mount snapshot location: + + ```bash + mount UUID= -o subvol=snapshots + ``` + +2. Create snapshots for the desired subvolumes: + + ```bash + btrfs subvolume snapshot / "/root/" + btrfs subvolume snapshot /home "/home/" + btrfs subvolume snapshot "//" + ``` + +3. Unmount after creating snapshots: + + ```bash + umount + ``` + +## Recovery + +1. **Mount a Subvolume with Recovery Options:** + + ```bash + mount -o recovery,subvol= UUID= + ``` + +2. **Clear Cache During Mount:** + + ```bash + mount -o clear_cache,subvol= UUID= + ``` + +3. **Data Restoration with btrfs restore:** + + ```bash + btrfs restore -D ``` diff --git a/notes/linux.md b/notes/linux.md index 9630b00..ec45a19 100644 --- a/notes/linux.md +++ b/notes/linux.md @@ -7,7 +7,6 @@ - [System Information](#system-information) - [Hardware Information](#hardware-information) - [Software Information](#software-information) -- [Commands to Get Information About Linux Version, Kernel Version, and Release](#commands-to-get-information-about-linux-version-kernel-version-and-release) - [User Management](#user-management) - [User Information](#user-information) - [Super User Management](#super-user-management) @@ -15,8 +14,6 @@ - [System Management](#system-management) - [Change password of a tar/openssl archive](#change-password-of-a-taropenssl-archive) - [Verify two possibly identical folders recursively](#verify-two-possibly-identical-folders-recursively) - - [USB Devices](#usb-devices) - - [Test USB Key](#test-usb-key) - [Diagnosis](#diagnosis) - [Debian Upgrade Issues](#debian-upgrade-issues) - [Wayland Issues](#wayland-issues) @@ -33,9 +30,6 @@ To gather detailed information about your hardware, use the following commands: - **`hwinfo`**: Offers detailed information about hardware components and can be more verbose than `lshw`. - **`lsscsi`**: Lists SCSI devices, including disks and other SCSI-attached hardware. - **`lsusb`**: Shows information about USB devices connected to your system. -- **`lsblk`**: Lists block devices such as hard drives and their partitions. -- **`df -H`**: Displays disk space usage in a human-readable format. -- **`fdisk -l`**: Lists all partitions on the system. - **`dmidecode`**: Retrieves hardware information from the BIOS. Use: - `dmidecode -t processor` for CPU details - `dmidecode -t memory` for RAM details @@ -45,13 +39,11 @@ To gather detailed information about your hardware, use the following commands: **Finding information on the Linux distribution** -# Commands to Get Information About Linux Version, Kernel Version, and Release - - **`lsb_release -a`**: Displays detailed information about the Linux distribution, including the distributor ID, description, release number, and codename. - **`cat /etc/debian_version`**: Displays the version of the Debian distribution if you're running a Debian-based system (like Ubuntu). - **`cat /etc/os-release`**: Displays information about the operating system, such as the name, version, and ID of the distribution. - **`cat /etc/*release`**: Searches for any files in the `/etc/` directory that contain the word `release` and displays their contents. This typically includes more detailed distribution information. -- **`cat /etc/*version`**: Similar to `cat /etc/*release`, but it looks for files containing the word `version`. It can provide additional version-related details. +- **`cat /etc/*version`**: Similar to `cat /etc/*release`, but looks for files containing the word `version`. It can provide additional version-related details. - **`hostnamectl`**: Displays system information related to the hostname and other metadata about the system. This may include the operating system, kernel version, and architecture. **Finding Path to Binary** @@ -59,17 +51,17 @@ To gather detailed information about your hardware, use the following commands: To find the location of an executable binary, use: ```bash -type composer +type ``` -This command will show the path to the `composer` executable if it's available in your `PATH`. +This command will show the path to a binary executable, ex `composer`, if it's available in your `PATH`. **Number of Words in a File** To count the number of words in a file, use: ```bash -wc filepath +wc ``` This command will show the number of words along with other details like lines and characters. @@ -79,11 +71,9 @@ This command will show the number of words along with other details like lines a To count the number of lines in a file, use: ```bash -wc -l filepath +wc -l ``` -This command will display the number of lines in the specified file. - ## User Management ### User Information @@ -93,25 +83,23 @@ This command will display the number of lines in the specified file. This variation of the adduser command uses the --gecos option to pre-fill the user's information (Full name, Room number, Work Phone, Home Phone, and Email) non-interactively, allowing you to automate user creation with predefined details. ```bash -adduser --gecos "Fabrice Quenneville,,,fabrice@fabq.ca" fabrice +adduser --gecos ",,," ``` -This variation of the adduser command creates a system user named "aptly" with a Bash shell, no password login (--disabled-password), a specified home directory (/home/aptly), and adds the user to a new group, while using the --gecos option to set the full name as "Aptly repository". +This variation creates a system user with a Bash shell, no password login (--disabled-password), a specified home directory (`/home/`), and adds the user to a new group, while using the --gecos option to set the full name as ``. ```bash -adduser --system --shell /bin/bash --gecos 'Aptly repository' --group --disabled-password --home /home/aptly aptly +adduser --system --shell /bin/bash --gecos '' --group --disabled-password --home /home/ ``` **List Users** -To list all users from the `/etc/passwd` file, use: +To list all users from the `/etc/passwd` file in alphabetical order, use: ```bash awk -F':' '{ print $1}' /etc/passwd | sort ``` -This command extracts the usernames from the `/etc/passwd` file and sorts them in alphabetical order. - ### Super User Management **Disable Root Login** @@ -124,7 +112,7 @@ To disable root login via SSH, perform the following steps: nano /etc/ssh/sshd_config ``` - Comment out the line containing `PermitRootLogin` by adding a `#` at the beginning of the line. + Comment out the line containing `PermitRootLogin`. 2. **Change Shell for Root User:** @@ -132,34 +120,36 @@ To disable root login via SSH, perform the following steps: nano /etc/passwd ``` - Find the line starting with `root` and change `/bin/bash` to `/sbin/nologin` to disable login for the root user. - - Save and close the file. Restart the SSH service for changes to take effect: + Find the line starting with `root` and change `/bin/bash` to `/sbin/nologin`. ```bash systemctl restart ssh ``` -**Sudo Management** - **Add User to Sudo Group** ```bash -adduser fabrice sudo +adduser sudo ``` **Update Sudoers File to Remove Password Requirement** -Edit the sudoers file: +Edit the sudoers file with the default editor: ```bash visudo ``` +Edit the sudoers file with `nano`: + +```bash +EDITOR=nano visudo +``` + Add the following line to allow the user to execute commands without a password: ```bash -fabrice ALL=(ALL) NOPASSWD:ALL + ALL=(ALL) NOPASSWD:ALL ``` ### Switch User @@ -167,82 +157,74 @@ fabrice ALL=(ALL) NOPASSWD:ALL **Switch to Another User as Sudoer** ```bash -sudo -i -u postgres +sudo -i -u ``` -This command switches to the `postgres` user with sudo privileges. - **Switch to Another User as Root** ```bash -su - postgres +su - ``` -This command switches to the `postgres` user with root privileges. - **Run command as specific user** ```bash -sudo -u www-data somecommand and arguments +sudo -u ``` **Change shell of a user** ```bash -chsh -s /bin/bash www-data -chsh -s /usr/sbin/nologin www-data +chsh -s /bin/bash +chsh -s /usr/sbin/nologin ``` **Change user with specific shell** ```bash -sudo -u www-data bash +sudo -u bash ``` ## System Management **Ensure hostname or add alias** -Set or update the hostname for your server. - ```bash nano /etc/hosts # Add the hostname alias: -# 127.0.1.1 local.servername.domain.com +# 127.0.1.1 nano /etc/hostname # Set the main hostname: -# 127.0.1.1 servername.domain.com servername +# 127.0.1.1 -hostnamectl set-hostname servername.domain.com +hostnamectl set-hostname ``` **Tar backup for a large number of small files** -These commands create backups using `tar` and transfer them securely over SSH. - Create a tar archive and transfer it to a remote server: ```bash -tar -c /path/to/dir | ssh fabrice@servername.domain.com 'tar -xvf - -C /absolute/path/to/remotedir' +tar -c /path/to/dir | ssh @ 'tar -xvf - -C /absolute/path/to/remotedir' ``` -Compress and transfer a folder, then store it as a .tar.gz file: +Compress and transfer a folder, then store it as a `.tar.gz` file: ```bash -tar zcvf - /folder | ssh fabrice@servername.domain.com "cat > /backup/folder.tar.gz" +tar zcvf - /folder | ssh @ "cat > /backup/folder.tar.gz" ``` -Transfer a compressed .tar.gz file and extract it on the remote server: +Transfer a compressed `.tar.gz` file and extract it on the remote server: ```bash -cat folder.tar.gz | ssh fabrice@servername.domain.com "tar zxvf -" +cat folder.tar.gz | ssh @ "tar zxvf -" ``` -Alternative method: change directory on the remote server before extracting: +Alternative: change directory on the remote server before extracting: ```bash -cat folder.tar.gz | ssh fabrice@servername.domain.com "cd /path/to/dest/; tar zxvf -" +cat folder.tar.gz | ssh @ "cd /path/to/dest/; tar zxvf -" ``` **List time zones** @@ -266,15 +248,13 @@ Alternatively, manually set the time zone by linking the correct file: ```bash mv /etc/localtime /etc/localtime-old -ln -s /usr/share/zoneinfo/America/Toronto /etc/localtime +ln -s /usr/share/zoneinfo/ /etc/localtime ``` **Find a specific service** -Search for a specific service running on your system. - ```bash -systemctl list-units --type=service | grep php +systemctl list-units --type=service | grep ``` ### Change password of a tar/openssl archive @@ -283,66 +263,61 @@ systemctl list-units --type=service | grep php To decrypt an `openssl`-encrypted archive using a password stored in a file: -1. **Store your password in this file.** - ```bash - nano $HOME/xyz001.txt - ``` -2. **Decrypt the archive** - Decrypt the archive using the password stored in xyz001.txt. +1. **Store your password in a temporary file:** ```bash - openssl aes-256-cbc -d -pbkdf2 -in servername-backup.tar.gz -out servername-backup.tar -pass file:$HOME/xyz001.txt + nano $HOME/ ``` -3. **Re-encrypt the archive with a new password** +2. **Decrypt the archive:** ```bash - nano $HOME/xyz001.txt - openssl aes-256-cbc -e -pbkdf2 -in servername-backup.tar -out servername-backup-new.tar.gz -pass file:$HOME/xyz001.txt - rm $HOME/xyz001.txt + openssl aes-256-cbc -d -pbkdf2 -in .tar.gz -out .tar -pass file:$HOME/ + ``` + +3. **Re-encrypt the archive with a new password:** + + ```bash + nano $HOME/ + openssl aes-256-cbc -e -pbkdf2 -in .tar -out -new.tar.gz -pass file:$HOME/ + rm $HOME/ ``` **Decode / Extract** -To decrypt and extract the contents of an encrypted archive directly into a directory: +Decrypt and extract the contents directly into a directory: ```bash -nano $HOME/xyz001.txt -openssl aes-256-cbc -d -pbkdf2 -in servername-backup.tar.gz -pass file:xyz001.txt | tar xz -C . -rm $HOME/xyz001.txt +nano $HOME/ +openssl aes-256-cbc -d -pbkdf2 -in .tar.gz -pass file: | tar xz -C . +rm $HOME/ ``` ### Verify two possibly identical folders recursively **With `diff`** -Check for differences between two directories, comparing all files recursively: - ```bash -diff -r servername-files/data/servername-repositories/ servername-repositories/ +diff -r / / ``` -Outputs any differences found between the two directories. - **With `rsync`** -Use `rsync` to show differences without copying any data: +Dry run — shows differences without copying any data: ```bash -rsync -avn servername-files/data/servername-repositories/ servername-repositories/ +rsync -avn / / ``` -- The `-n` flag means this is a dry run, which won’t make any changes. +- `-n`: dry run, no changes made. **With `cmp`** -This script compares files in two directories and identifies any differences between matching file names. - ```bash #!/bin/bash -dir1="servername-files/data/servername-repositories/" -dir2="servername-repositories/" +dir1="/" +dir2="/" # Check if both directories exist before proceeding. if [ ! -d "$dir1" ] || [ ! -d "$dir2" ]; then @@ -361,149 +336,12 @@ for file1 in $(find "$dir1" -type f); do done ``` -## USB Devices - -### Test USB Key - -**Device Information** - -Check if the system recognizes the device and show the latest system messages related to USB devices being connected: - -```bash -lsusb -dmesg | tail -n 20 -``` - -**Find Mount Points and Device Information** - -Identify mount points, partitions, and other relevant details of mounted devices: - -```bash -lsblk -f -df -h | grep /dev/sdc -findmnt /dev/sdc1 -mount | grep /dev/sd -``` - -**Print Detailed Information About the USB Key** - -View detailed partition and disk information: - -```bash -fdisk -l /dev/sdc -``` - -**Test the File System** - -Check and repair the filesystem on the USB key: - -```bash -fsck /dev/sdc1 -``` - -**Test Data Integrity** - -Perform read/write tests to ensure the integrity of the USB key: - -1. **Unmount the USB Key** (if mounted): - - ```bash - umount /media/fabrice/BD48-F8BB - ``` - -2. **Write Test**: - - ```bash - dd if=/dev/zero of=/dev/sdc bs=4M count=256 status=progress - ``` - -3. **Read Test**: - - ```bash - dd if=/dev/sdc of=/dev/null bs=4M count=256 status=progress - ``` - -**Check for Bad Blocks** - -Identify any bad sectors on the USB key: - -- **Read-only test**: - - ```bash - badblocks -v /dev/sdc - ``` - -- **Non-destructive read-write test**: - - ```bash - badblocks -nsv /dev/sdc - ``` - - - The `-n` option performs a non-destructive read-write test. - - The `-s` option shows progress. - - The `-v` option is for verbose output. - -**Perform a SMART Test** - -Run SMART diagnostics to test the health of the USB key: - -1. **Start a short SMART test**: - - ```bash - smartctl -t short /dev/sdc - ``` - -2. **View test results**: - - ```bash - smartctl -a /dev/sdc - ``` - -**Benchmark the Speed** - -Measure the read speed of the USB key: - -```bash -hdparm -t /dev/sdc -``` - -**Unmount and Safely Remove** - -Unmount the USB key and safely remove it from the system: - -```bash -umount /mnt/usb -eject /dev/sdc -``` - -**Switching two USB keys** - -The following commands copy data between two USB drives, format one of them, and restore the data. - -```bash -cp -r /media/fabrice/465A-759B "/tmp/Michael Allison" -umount /dev/sdc1 -mkfs.vfat /dev/sdc1 - -umount /dev/sdc1 -dd if=/dev/sdc of=/tmp/usb_image.img bs=4M status=progress -mkfs.vfat /dev/sdc1 - -cp -r "/tmp/Michael Allison" /media/fabrice/D67D-ADF8 -umount /dev/sdc1 - -dd if=/tmp/usb_image.img of=/dev/sdc bs=4M status=progress -sync -``` - ## Diagnosis ### Debian Upgrade Issues **Apt Logs** -View the APT logs to check for package installation and updates history: - ```bash less /var/log/apt/history.log ``` @@ -533,19 +371,19 @@ journalctl -b | grep -i "gnome-shell" **Download and Install Fonts** -1. **Download the Font Archive**: +1. **Download the Font Archive:** ```bash - wget https://path/to/font/archive.tar.gz + wget https:// ``` -2. **Extract the Font Files**: +2. **Extract the Font Files:** ```bash - tar -xzvf font-archive.tar.gz + tar -xzvf .tar.gz ``` -3. **Copy the Font Files to the Local Fonts Directory**: +3. **Copy the Font Files:** **Local font directory** @@ -567,14 +405,12 @@ journalctl -b | grep -i "gnome-shell" **Update the Font Cache** -**Force a Reload of the Installed Font Cache**: - ```bash sudo su - fc-cache -fv fc-cache -frv ``` -- **`-f`**: Force re-generation of apparently up-to-date cache files, overriding the timestamp checking. +- **`-f`**: Force re-generation of cache files, overriding timestamp checking. - **`-r`**: Erase all existing cache files and rescan. - **`-v`**: Display status information while busy. diff --git a/notes/storage.md b/notes/storage.md new file mode 100644 index 0000000..39abaf0 --- /dev/null +++ b/notes/storage.md @@ -0,0 +1,314 @@ +# Storage + +## Table of Contents + +- [Storage](#storage) + - [Table of Contents](#table-of-contents) + - [Placeholders](#placeholders) + - [Drive Information](#drive-information) + - [List Block Devices](#list-block-devices) + - [List Mountpoints and Usage](#list-mountpoints-and-usage) + - [Inspect fstab](#inspect-fstab) + - [Find Device Path from UUID](#find-device-path-from-uuid) + - [Power On Hours](#power-on-hours) + - [Partitions and Filesystems](#partitions-and-filesystems) + - [Mounting](#mounting) + - [SMART Diagnostics](#smart-diagnostics) + - [Badblocks](#badblocks) + - [Cloning drives and images with dd](#cloning-drives-and-images-with-dd) + - [Benchmarking](#benchmarking) + - [USB Devices](#usb-devices) + - [Device Recognition](#device-recognition) + - [Test Device Integrity](#test-device-integrity) + - [Switching Two USB Keys](#switching-two-usb-keys) + +## Placeholders + +Replace the placeholders below with the appropriate values for your setup: + +- **Devices** + - Block device: `` (e.g., /dev/sda) + - Partition: `` (e.g., /dev/sda1) + - UUID: `` (e.g., a1b2c3d4-e5f6-7890-abcd-ef1234567890) + +- **Paths** + - Mount point: `` (e.g., /mnt/media) + - Directory path: `` (e.g., /mnt/data) + +## Drive Information + +### List Block Devices + +Check all attached block devices by type: + +```bash +ls /dev/sd* +ls /dev/nv* +``` + +### List Mountpoints and Usage + +**Tree view of all block devices with sizes and mountpoints** + +```bash +lsblk +lsblk -f +``` + +- `-f`: Add filesystem type, label, and UUID to the tree. + +**Disk space usage for all mounted filesystems** + +```bash +df -h +``` + +- `-h`: Human-readable. + +**Disk space usage for a specific device** + +```bash +df -h | grep +``` + +### Inspect fstab + +**View all configured mount entries** + +```bash +cat /etc/fstab +``` + +### Find Device Path from UUID + +Using `lsblk`: + +```bash +lsblk -o NAME,UUID,MOUNTPOINT +``` + +Using `blkid`: + +```bash +blkid | grep +blkid -U +``` + +### Power On Hours + +Check power-on hours across multiple drives at once: + +```bash +for dev in /dev/sd{a,b,c,d}; do echo -n "$dev: "; smartctl -a $dev | grep "Power_On_Hours"; done +``` + +## Partitions and Filesystems + +**View partition table and disk details:** + +```bash +fdisk -l +``` + +**Check and repair a filesystem:** + +```bash +fsck +``` + +## Mounting + +**Validate all `fstab` entries** + +```bash +findmnt --verify --verbose +``` + +**Dry-run mount of all fstab entries, reports what would succeed or fail** + +```bash +mount -fav +``` + +- `-f`: Fake mount — goes through the motions without making the actual syscall. +- `-a`: Process all entries in `/etc/fstab` (excluding `noauto`). +- `-v`: Verbose output. + +**Find mount point details:** + +```bash +findmnt +mount | grep +``` + +**Apply fstab changes without rebooting** + +```bash +mount -o remount -a +``` + +**Unmount and Safely Remove** + +```bash +umount +eject +``` + +## SMART Diagnostics + +**Get full SMART information for a drive:** + +```bash +smartctl -a +``` + +**Get drive identity and model info only:** + +```bash +smartctl -i +``` + +**Run a short SMART test:** + +```bash +smartctl -t short +``` + +**Run a long SMART test:** + +```bash +smartctl -t long +``` + +**View test results:** + +```bash +smartctl -a +``` + +**Check SMART device statistics (error counters, etc.):** + +```bash +smartctl -A +``` + +## Badblocks + +**Read-only test:** + +```bash +badblocks -v +``` + +**Non-destructive read-write test:** + +`Save to RAM -> Write Pattern -> Verify -> Restore` + +```bash +badblocks -nsv +``` + +- `-n`: Non-destructive read-write test. +- `-s`: Show progress. +- `-v`: Verbose output. +- Warning: If the computer loses power or the kernel panics after Step 2 but before Step 4, that specific block of data will be lost + +**Destructive write test** + +Overwrites all data — use only on blank drives or drives to be deleted. + +```bash +badblocks -wsv +``` + +## Cloning drives and images with dd + +**Clone a drive or create an image:** + +```bash +dd if= of= bs=4M status=progress +dd if= of=.img bs=4M status=progress +``` + +**Restore from image:** + +```bash +dd if=.img of= bs=4M status=progress +sync +``` + +## Benchmarking + +**Measure read speed with `hdparm`:** + +```bash +hdparm -t +``` + +## USB Devices + +### Device Recognition + +Check if the system recognizes the device and show recent USB-related kernel messages: + +```bash +lsusb +dmesg | tail -n 20 +``` + +### Test Device Integrity + +Warning: The write test is destructive. All data on will be permanently overwritten. Verify the target device with lsblk before running. + +1. **Unmount the device** (if mounted): + + ```bash + umount + ``` + +2. **Write test:** + + ```bash + dd if=/dev/zero of= bs=4M count=256 status=progress + ``` + +3. **Read test:** + + ```bash + dd if= of=/dev/null bs=4M count=256 status=progress + ``` + +4. **Check for bad blocks:** + + ```bash + badblocks -wsv + ``` + +5. **Run a SMART test:** + + ```bash + smartctl -t short + smartctl -a + ``` + +### Switching Two USB Keys + +Copy data off, reformat, and restore: + +```bash +cp -r /media// /tmp/ +umount +mkfs.vfat + +cp -r /tmp/ /media// +umount +``` + +Or clone with `dd`: + +```bash +dd if= of=/tmp/usb_image.img bs=4M status=progress +mkfs.vfat +dd if=/tmp/usb_image.img of= bs=4M status=progress +sync +```