add syntax highlight

tengel 2024-09-05 08:27:27 -05:00
parent d34b1dde6e
commit 9162b14e6b
25 changed files with 82 additions and 83 deletions

@ -1,6 +1,6 @@
Combine repos in git
```
```bash
# Create a new empty repo and clone it ("origin")
git clone git@gitlab.com:${myname}/newproject.git
cd newproject/
@ -17,7 +17,7 @@ git subtree add --prefix=oldproject_2 \
# Do any cleanup needed - for example getting rid of a submodule
git rm oldproject_1/somegitsubmodule
git rm oldproject_1/.gitmodules
git rm oldproject_1/.gitmodules
git commit -m "remove stale submodule" \
oldproject_1/.gitmodules \
oldproject_1/somegitsubmodule

@ -20,7 +20,7 @@ scenario:
Test each account to ensure the "welcome!" message reflect the correct user:
```
```console
$ ssh -i ~/.ssh/id_rsa_user1 -T git@gitlab.com
Welcome to GitLab, @user1!
@ -30,13 +30,13 @@ Welcome to GitLab, @user2!
Within a cloned repository, locally configure that repository to use a specific key:
```
```console
$ git clone git@gitlab.com:user1/foo.git
$ cd foo
$ git config core.sshCommand "ssh -i ~/.ssh/id_rsa_user1 -F /dev/null"
```
and/or
```
```console
$ git clone git@gitlab.com:user2/bar.git
$ cd bar
$ git config core.sshCommand "ssh -i ~/.ssh/id_rsa_user2 -F /dev/null"
@ -44,7 +44,7 @@ $ git config core.sshCommand "ssh -i ~/.ssh/id_rsa_user2 -F /dev/null"
This adds the setting to the local `.git/config` file like so:
```
```ini
[core]
sshCommand = ssh -i ~/.ssh/id_user73 -F /dev/null
...
@ -54,7 +54,7 @@ This adds the setting to the local `.git/config` file like so:
If the initial clone requires a specific SSH key before the above is available, it must be exported to the environment first.
```
```bash
GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa_user99 -F /dev/null" \
git clone git@gitlab.com:user99/privatestuff.git
```

@ -1,6 +1,6 @@
Rewrite git history to change a username/email
```
```bash
#!/usr/bin/env bash
#
# Rewrite git commit history to replace user/email values

@ -19,7 +19,7 @@ Gitea allows the same SSH keypair to be used as Deploy Keys for multiple reposit
One-time need, prepare the initial repository sync setup and test the initial mirroring.
**buildmirrors.sh**
```
```bash
#!/usr/bin/env bash
WDIR="${HOME}/repos/mirror"
@ -54,7 +54,7 @@ done
To be used to keep the repos mirrored over time, whether automated or manual.
**repomirror.sh**
```
```bash
#!/usr/bin/env bash
WDIR="${HOME}/repos/mirror"

@ -1,6 +1,6 @@
Allow unattended passphrases to work, stop a GUI dialog when using a CLI app:
```
```console
$ cat ~/.gnupg/gpg-agent.conf
pinentry-program /usr/bin/pinentry-tty
@ -22,7 +22,7 @@ diff -uN passwords_backup.csv passwords_restored.csv
```
Use a keyserver with Let's Encrypt wrapped around it
```
```console
$ cat ~/.gnupg/dirmngr.conf
hkp-cacert /home/FOO/.gnupg/le.pem
@ -30,7 +30,7 @@ keyserver hkps://keys.openpgp.org
```
Set a default key to use
```
```console
$ cat ~/.gnupg/gpg.conf
default-key ABCD1234

@ -66,7 +66,7 @@ CopyArrivalDate yes
This script leverages healthchecks.io to alert on failure; replace XXXXX with the UUID of your monitor URL.
**${HOME}/bin/hasync.sh**
```
```bash
#!/bin/bash
# vars

@ -1,6 +1,6 @@
Default IPv4 gateway out of `ip`
```
```bash
#!/bin/sh
# get ipv4 default gateway with ip / grep

@ -1,6 +1,6 @@
IPv4 address in pure bash/zsh
```
```bash
[[ -n "$1" ]] || exit 1
IP=$(ip -4 -o addr show dev $1 primary 2>/dev/null)
IP=${IP%%/*}

@ -1,6 +1,6 @@
Decompress a `vmlinuz` manually by finding the magic offset
```
```console
# od -t x1 -A d vmlinuz | grep "1f 8b 08 00"
0018352 1c ff ff ff 1f 8b 08 00 00 00 00 00 02 03 ec dd

@ -28,7 +28,7 @@ vim /etc/fstab
Manual LUKS partition opened after boot (remote SSH)
**opendata.sh**
```
```bash
#!/usr/bin/env bash
#
# /dev/sda3 -> /data
@ -72,7 +72,7 @@ fi
Manual LUKS partition close
**closedata.sh**
```
```bash
#!/usr/bin/env bash
_LUKS=cdata

@ -14,8 +14,8 @@ parted ./gpt.dat mkpart primary ext3 2048s 100%
hexdump -C msdos.dat > msdos_part.hex
hexdump -C gpt.dat > gpt_part.hex
sudo losetup -P /dev/loop0 ./msdos.dat
sudo losetup -P /dev/loop1 ./gpt.dat
sudo losetup -P /dev/loop0 ./msdos.dat
sudo losetup -P /dev/loop1 ./gpt.dat
sudo pvcreate /dev/loop0p1
sudo pvcreate /dev/loop1p1
sudo losetup -d /dev/loop0

@ -1,6 +1,6 @@
Pull out mp3 tags to create a libre.fm import file
```
```bash
#!/usr/bin/env bash
DTB=1251499494 # epoch start time

@ -1,6 +1,6 @@
DNS records to route all email (spam) to honeypots
```
```zone
# dns records for no-email domains
## https://www.gov.uk/guidance/protect-domains-that-dont-send-email

@ -11,7 +11,7 @@ find /mnt/d11/ -name cloud_inet6
# /mnt/d11/etc/network/if-post-down.d/cloud_inet6
# /mnt/d11/etc/network/if-pre-up.d/cloud_inet6
umount /mnt/d11
umount /mnt/d11
qemu-nbd --disconnect /dev/nbd0
modprobe -r nbd
```

@ -41,6 +41,6 @@ net.ipv4.tcp_mtu_probing=1
sunrpc.tcp_slot_table_entries=128
sunrpc.udp_slot_table_entries=128
## Also add to /etc/rc.local (10G NIC):
## Also add to /etc/rc.local (10G NIC):
## /sbin/ip link set eth4 txqueuelen 10000
```

@ -1,6 +1,6 @@
One-liner replacements for basic netcat port pings (`-tzv`)
```
```bash
# if netcat is not available
# - requires --enable-net-redirections compiled in with bash
# - works on RHEL/CentOS/Fedora but not Ubuntu/Debian

@ -15,13 +15,13 @@ Mission: create a virtual top-level directory for all users logging in, which is
Make the virtual directory mount point:
```
```console
# mkdir /vdir
```
Create a config for mounting the namespace, this will add the username onto the end of the directory; for example `/home/fred/vdirfred` - the idea is that the second field does _not_ have to be `$HOME`, it could be another directory like `/srv/vdirusers` such that each subdir gets a unique name dynamically like `/srv/vdirusers/vdirfrank` - see the man pages.
```
```console
# vi /etc/security/namespace.conf
/vdir $HOME/vdir user:iscript=userperms.sh
@ -54,7 +54,7 @@ Activate the new configuration - note that this edits PAM, the rule of thumb is
**IF YOU BREAK PAM, YOU LOCK EVERYONE OUT - INCLUDING `root`**
```
```console
# vi /etc/pam.d/common-session
session required pam_namespace.so ignore_instance_parent_mode
@ -64,7 +64,7 @@ session required pam_namespace.so ignore_instance_parent_mode
Log in as the user and test:
```
```console
$ ssh fred@server
server$ touch vdirfred/foo
@ -80,7 +80,7 @@ bar foo
As root, `/vdir` should remain empty (no user created files) as it's a virtual mount point:
```
```console
# ls /vdir
#
```

@ -17,7 +17,7 @@ Things to quickly do to the host:
2. Disconnect the non-functional Enterprise (requires subscription) APT repo and plug in the regular one:
```
# cat /etc/apt/sources.list.d/pve-enterprise.list
# cat /etc/apt/sources.list.d/pve-enterprise.list
#deb https://enterprise.proxmox.com/debian/pve stretch pve-enterprise
deb http://download.proxmox.com/debian/pve stretch pve-no-subscription
```
@ -28,7 +28,7 @@ Run a standard apt-get dist-upgrade at this point and reboot. Host Networking br
The installer defaults creates a LVM thin pool "data" out of the OS disks - this gets deleted, and a new one created over on the secondary RAID-10 at /dev/sdb:
```
```console
# vi /etc/pve/storage.cfg
Delete the stanza referring to the default "lvmthin" data LV

@ -29,7 +29,7 @@ SUBSYSTEM=="drm", DRIVERS=="radeon", ATTR{device/power_dpm_force_performance_lev
Helper script, nothing fancy.
**radcheck.sh**
```
```bash
#!/usr/bin/env bash
_MOD="radeon"

@ -71,7 +71,7 @@ The content of files is stored in contiguous blocks of 4096 bytes (the actual si
The actual block size, as well as the actual fragment size, are stored in the superblock and can be retrieved with the option `--superblock`. For example,
```
```console
$ ext3grep $IMAGE --superblock | grep 'size:'
Block size: 4096
Fragment size: 4096
@ -81,20 +81,20 @@ Here `IMAGE` is an environment variable that was set to the name of the device (
The entire partition is divided into an integral number of blocks, starting to count at 0. Thus, if you ever want to make a copy of block number N, you could do:
```
```console
$ dd if=$IMAGE bs=4096 count=1 skip=$N of=block.$N
```
Where N runs from 0 till (but not including) the number of blocks as stored in the superblock. For example,
```
```console
$ ext3grep $IMAGE --superblock | grep 'Blocks count:'
Blocks count: 2441824
```
Having any block number, one can print information about it by using the command line option `--block`. For example,
```
```console
$ ext3grep $IMAGE --ls --block 600
[...]
Group: 0
@ -122,7 +122,7 @@ The superblock isn't really a block. It's size is always 1024 bytes and the firs
One could read the contents of the first superblock with dd as follows:
```
```console
$ dd if=$IMAGE bs=1024 skip=1 count=1 of=superblock
```
@ -187,16 +187,16 @@ The C-struct for the superblock is given in the header file `/usr/include/linux/
Each ext3 file system is devided into groups, with a fixed number of blocks per group, except the last group which contains the remaining blocks. The number of blocks per group is given in the superblock, ie
```
```console
$ ext3grep $IMAGE --superblock | grep 'Blocks per group'
# Blocks per group: 32768
Blocks per group: 32768
```
Each group uses one block as a bitmap to keep track of which block inside that group is allocated (used); thus, there can be at most 4096 \* 8 = 32768 normal blocks per group.
Another block is used as bitmap for the number of allocated inodes. Inodes are data structures of 128 bytes (they can be extended in theory; the real size is given in the superblock once again) that are stored in a table, (4096 / 128 = 32 inodes per block) in each group. Having at most 32768 bits in the bitmap, we can conclude that there will be at most 32768 inodes per group, and thus 32768 / 32 = 1024 blocks in the inode table of each group. The actual size of the inode table is given by the actual number of inodes per group, which is also stored in the superblock.
```
```console
$ ext3grep $IMAGE --superblock | egrep 'Size of inode|inodes per group'
Number of inodes per group: 16288
Size of inode structure: 128
@ -220,7 +220,7 @@ Since the size of this struct is padded to a power of 2, 32 bytes, there fit pre
The content of the table is printed by `ext3grep` if no action or group is specified on the command line. For example,
```
```console
$ ext3grep $IMAGE
No action specified; implying --superblock.
[...]
@ -280,14 +280,14 @@ Note that this index also determines the corresponding bit in the inodes bitmap.
As such, groups have been made transparent: every inode can be addressed with a number in the contiguous range `[1, number_of_inodes]`, where the number of inodes is given by:
```
```console
$ ext3grep $IMAGE --superblock | grep 'Inodes count'
Inodes count: 1221600
```
In some case you might want to know which block in the file system belongs to the inode table that stores a particular inode. This can be retrieved with the command line option `--inode-to-block`, for example:
```
```console
$ ext3grep $IMAGE --inode-to-block 2
[...]
Inode 2 resides in block 600 at offset 0x80.
@ -297,7 +297,7 @@ Inode number 2 (the macro `EXT3_ROOT_INO` in `ext3_fs.h`) is always used for the
Having the inode number, one can print it's contents with `ext3grep`, for example:
```
```console
$ ext3grep $IMAGE --inode 2 --print
Number of groups: 75
Loading group metadata... done
@ -368,7 +368,7 @@ Table 4. A Directory Entry
Using the options `--ls --inode $N`, `ext3grep` lists the contents of each directory block of inode N. For example, to list the root directory of a partition:
```
```console
$ ext3grep $IMAGE --ls --inode 2
Number of groups: 75
Loading group metadata... done
@ -397,7 +397,7 @@ Note that `ext3grep` prints all directory entries, deleted not. There are two wa
Because `ext3grep` prints also deleted entries, it is very well possible that the SAME entry occurs multiple times. In particular, if a file is moved, a duplicate remains that will still be visible. Ie,
```
```console
$ ext3grep $IMAGE --ls --inode 195457 | grep '\.viminfo$'
7 8 r 201434 D 1202351096 Thu Feb 7 03:24:56 2008 rrw-r--r-- .viminfo
18 19 r 195995 D 1202351097 Thu Feb 7 03:24:57 2008 rrw------- .viminfo
@ -469,7 +469,7 @@ Secondly, you have to realize that only the inode number, the file type in the t
It is possible to apply filters to the output of `--ls`. An overview of the available filters is given in the output of the `--help` option:
```
```console
$ ext3grep $IMAGE --help
[...]
Filters:
@ -494,7 +494,7 @@ Filters:
In order to easily determine sensible values for `--after` and `--before` the action `--histogram=dtime` was added. This command line option causes `ext3grep` to print a histogram of time versus number of deleted inodes. If you delete a large number of files at once, for example with `rm -rf`, then it should be easy to determine a time window within which the deletion took place. For example, here I zoomed in on my personal disaster where I deleted a little over fifty thousand files from my home directory:
```
```console
$ ext3grep $IMAGE --histogram=dtime --after=1202351086 --before=1202351129
Only show/process deleted entries if they are deleted on or after Thu Feb 7 03:24:46 2008 and before Thu Feb 7 03:25:29 2008.
@ -560,14 +560,14 @@ It is important to set a good value for `--after` before recovering all files, o
The journal is a file existing of a fixed number of blocks. It's inode is `EXT3_JOURNAL_INO`, which is usually 8. The actual inode can also be found in the superblock:
```
```console
$ ext3grep $IMAGE --superblock | grep 'Inode number of journal file'
Inode number of journal file: 8
```
and the size then can be found by printing inode 8:
```
```console
$ ext3grep $IMAGE --print --inode 8
Number of groups: 75
Loading group metadata... done
@ -610,7 +610,7 @@ where you can see that the size of my journal is 134217728 bytes, or 32768 block
The first block of the journal file (block 1115 in the above example) contains the 'journal superblock'. It's structure is defined in `/usr/include/linux/jbd.h` as `journal_superblock_t`. It can be printed with:
```
```console
$ ext3grep $IMAGE --journal --superblock
Journal Super Block:
@ -642,7 +642,7 @@ Number of descriptors in journal: 30258; min / max sequence numbers: 4379495 / 4
Here you can see that the journal actually starts in Journal Block Number 1, and the last block is Journal Block Number 32768. These are thus not the same as the file system block numbers. One can find the real block number with, for example,
```
```console
$ ext3grep $IMAGE --journal --journal-block 1
[...]
Group: 0
@ -664,7 +664,7 @@ In the following example we will manually recover a small file. Only partial out
Using `ext3grep $IMAGE --ls --inode` we find the name of the file that we want to recover:
```
```console
$ ext3grep $IMAGE --ls --inode 2 | grep carlo
3 end d 195457 D 1202352103 Thu Feb 7 03:41:43 2008 drwxr-xr-x carlo
@ -677,7 +677,7 @@ $ ext3grep $IMAGE --ls --inode 309540 | grep start_azureus
Obviously, inode 309631 is erased and we have no block numbers for this file:
```
```console
$ ext3grep $IMAGE --print --inode 309631
[...]
Inode is Unallocated
@ -700,14 +700,14 @@ Direct Blocks:
Therefore, we will try to look for an older copy of it in the journal. First, we find the file system block that contains this inode:
```
```console
$ ext3grep $IMAGE --inode-to-block 309631 | grep resides
Inode 309631 resides in block 622598 at offset 0xf00.
```
Then we find all journal descriptors referencing block 622598:
```
```console
$ ext3grep $IMAGE --journal --block 622598
[...]
Journal descriptors referencing block 622598:
@ -735,7 +735,7 @@ This means that the transaction with sequence number 4381294 has a copy of block
If you try to print such a block, ext3grep recognizes that it's a block from an inode table and will print the contents of all 32 inodes in it. We only wish to see inode 309631 however; so we use a smart grep:
```
```console
$ ext3grep $IMAGE --print --block 8931 | grep -A15 'Inode 309631'
--------------Inode 309631-----------------------
Generation Id: 2771183319
@ -756,7 +756,7 @@ Direct Blocks:
This is indeed the same as we saw in block 622598. Next we look at smaller sequence numbers until we find one with a 0 Deletion time. The first one that we find (bottom up) is block 6073:
```
```console
$ ext3grep $IMAGE --print --block 6073 | grep -A15 'Inode 309631'
--------------Inode 309631-----------------------
Generation Id: 2771183319
@ -777,7 +777,7 @@ Direct Blocks: 645627
The above is automated and can be done much faster with the command line option `--show-journal-inodes`. This option will find the block that the inode belongs to, then finds all copies of that block in the journal, and subsequently prints only the requested inode from each of these block (each of which contains 32 inodes, as you know), eliminating duplicates:
```
```console
$ ext3grep $IMAGE --show-journal-inodes 309631
Number of groups: 75
Minimum / maximum journal block: 1115 / 35026
@ -821,7 +821,7 @@ Direct Blocks: 645627
The file is indeed small: only one block. We copy this block with dd as shown before:
```
```console
$ dd if=$IMAGE bs=4096 count=1 skip=645627 of=block.645627
1+0 records in
1+0 records out
@ -830,7 +830,7 @@ $ dd if=$IMAGE bs=4096 count=1 skip=645627 of=block.645627
and then edit the file to delete the trailing zeroes, or copy the first 40 bytes (the given size of the file):
```
```console
$ dd if=block.645627 bs=1 count=40 of=start_azureus
40+0 records in
40+0 records out
@ -845,7 +845,7 @@ Recovered\!
Note that it is possible to see all descriptors of a given transaction. The transaction that we used to recover this file was 4382098. The complete transaction can be seen with:
```
```console
$ ext3grep $IMAGE --journal-transaction 4382098
[...]
Prev / Current / Next sequences numbers: 4382097 4382098 4382099
@ -863,7 +863,7 @@ Of course, it would be annoying to recover larger files, existing of many blocks
No such care seems necessary to recover a single file, you can just pass it's path to `ext3grep`:
```
```console
$ ext3grep $IMAGE --restore-file carlo/bin/start_kvm
[...]
Restoring carlo/bin/start_kvm
@ -885,7 +885,7 @@ In order for this to work you will first have to pass stage1 and stage2 of the d
It is possible to dump all file names that `ext3grep` can find, using the command line option `--dump-names`:
```
```console
$ ext3grep $IMAGE --dump-names
carlo
carlo/.Trash
@ -913,7 +913,7 @@ Once you are satisfied with the output of `--dump-names`, you can replace `--dum
For example,
```
```console
$ time ext3grep $IMAGE --restore-all --after=1202351117
Only show/process deleted entries if they are deleted on or after Thu Feb 7 03:25:17 2008.
[...]
@ -940,7 +940,7 @@ The stage 1 cache file is written to `DEVICE.ext3grep.stage1`, where `DEVICE` is
The format of the stage1 cache file is:
```
```console
$ cat md5.ext3grep.stage1
# Stage 1 data for md5.
# Inodes and directory start blocks that use it for dir entry '.'.
@ -997,7 +997,7 @@ Because inodes are reused, it happens often that an old directory entry (of a de
In order to make it easier to clean these up, `ext3grep` provides the command line option `--show-hardlinks`.
```
```console
$ ext3grep $IMAGE --show-hardlinks
[...]
Inode 309562:
@ -1022,7 +1022,7 @@ The program has been written _while_ I was learning how ext3 works. It's earlies
All command line options are listed by providing `--help` on the command line:
```
```console
$ ext3grep $IMAGE --help
Usage: ext3grep [options] [--] device-file
Options:

@ -1,6 +1,6 @@
Quick and dirty, probably isn't super safe
```
```bash
#!/usr/bin/env bash
shopt -s globstar nullglob
while set -- **/*.zip; [ $# -ge 1 ] do

@ -96,7 +96,7 @@ RMAN> Crosscheck ARCHIVELOG ALL;
oracle query for list of the tablespaces
--------------------------------------
select tablespace_name, CONTENTS from dba_tablespaces;
select tablespace_name, CONTENTS from dba_tablespaces;
oracle query for list of datafiles
--------------------------------------
@ -107,12 +107,12 @@ https://docs.oracle.com/cd/B19306_01/server.102/b14237/dynviews_1076.htm#REFRN30
ASM disk timeouts
SQL> select name,value,describe from v$asm_hidden_paras;
NAME VALUE DESCRIBE
--------------------------------------- -------- ----------------------------------------------------------------------
_asm_acd_chunks 1 initial ACD chunks created
SQL> select name,value,describe from v$asm_hidden_paras;
NAME VALUE DESCRIBE
--------------------------------------- -------- ----------------------------------------------------------------------
_asm_acd_chunks 1 initial ACD chunks created
[...]
_asm_global_dump_level 267 System state dump level for ASM asserts
_asm_global_dump_level 267 System state dump level for ASM asserts
_asm_hbeatiowait 15 number of secs to wait for PST Async Hbeat IO return
_asm_hbeatwaitquantum 2 quantum used to compute time-to-wait for a PST Hbeat check
@ -140,8 +140,7 @@ MySQL real DB size by Engine:
select engine,count(*),sum(index_length+data_length)/1024/1024 from information_schema.tables group by engine;
MySQL table size of a single table:
SELECT table_name,`engine`,ROUND(data_length/1024/1024,2) total_size_mb,ROUND(index_length/1024/1024,2) total_index_size_mb, table_rows FROM
information_schema.TABLES WHERE table_schema = 'threadless' and table_name = 'users';
SELECT table_name,`engine`,ROUND(data_length/1024/1024,2) total_size_mb,ROUND(index_length/1024/1024,2) total_index_size_mb, table_rows FROM information_schema.TABLES WHERE table_schema = 'threadless' and table_name = 'users';
MySQL flush buffers for shutdown:
# mysql -e "flush logs;"

@ -1,6 +1,6 @@
Cut a disk in half based on sectors, skipping the first 2048
```
```bash
#!/usr/bin/env bash
DISK="/dev/xvdb"; OFFSET=2048 \

@ -10,7 +10,7 @@ Task: pause rclone, kill the in-progress TCP connection, resume rclone which wil
Find the process and it's TCP stalled connection:
```
```console
$ ps axo pid,comm | grep [r]clone
6874 rclone
@ -20,7 +20,7 @@ $ ss -ntp | grep [r]clone | awk '{print $5, $6}'
Pause the process, kill the connection (as root), resume the process:
```
```console
$ kill -STOP 6874
$ sudo ss -K dst 74.120.8.14 dport 443

@ -1,6 +1,6 @@
Google Play -> Youtube Music -> Spotlistr
```
```bash
#!/usr/bin/env bash
# build import list for spotlistr