Differences

This shows you the differences between two versions of the page.

--- quickreference:unix [2022/04/30 15:08] – [Create Swap file] rodolico
+++ quickreference:unix [2025/02/05 00:12] (current) – [Shell (mainly BASH)] rodolico
@@ Line 4: / Line 4: @@
 ===== Systems Administration =====
+==== Partitioning large drives ====
+Drives greater than 2 Terabytes are not handled well by the standard //fdisk// application, so instead we use parted. Fun Fact!!! gparted is a nice little GUI interface to this. But, we're dealing with command line stuff here.
+This assumes we have a drive, sdg, that we want to set up with gpt and create one partition on. That partition will set up on optimal sector boundries, and use all of the space available.
+<code bash>
+# remove all old file system information. Not necessary, but I do it just because I can
+wipefs -a /dev/sdg
+# make this a gpt disk. Will wipe out any other partitioning scheme
+parted /dev/sdg mklabel gpt
+# make a new partition on optimal sector boundries. This is a primary partition, and starts
+# at the beginning of the disk (0%) and goes to the end of the disk (100%)
+# I put that in quotes as, from what I've read, the percent symbol does not work well
+# within the bash command line
+# note, we are not telling it what file system to use, so it defaults to Linux
+parted -a optimal /dev/sdg mkpart primary '0%' '100%'
+# display the information on the disk
+parted /dev/sdg print
+# format as ext4, no reserved space, and a disk label marked 'backup'
+mkfs.ext4 -m0 -Lbackup /dev/sdg
+</code>
+==== Rapidly wipe multiple hard drives ====
+Nothing beats DBAN [https://dban.org/] in ease of use and a feeling of good security. However, I recently had an issue where I had a server with 7 slow hard disks containing data that really wasn't all that sensitive, so I simply wanted to put a bunch of zeros on it, so I booted off of my SystemRescueCD thumbdrive [https://www.system-rescue.org/] and ran the following bash script. Should work in any shell which has the //for// command, however.
+<code bash wipedrives.sh>
+#! /usr/bin/env bash
+# for truly not sensitive information, this command wipes all the OS information
+for drive in a b c d e f g
+do
+   wipefs -a /dev/sd$drive
+done
+# but, to really remove in a way that takes tons of effort to recover, do this also
+for drive in a b c
+do
+   echo Cleaning sd%drive
+   dd if=/dev/zero | pv -petrs 580G | dd of=/dev/sd$drive
+done
+</code>
+I had 7 drives to wipe, and this takes about 5 hours per drive, so a total of 35 hours. I realized I could probably run all 7 processes in parallel since, on my system, the drive controller is a lot faster than any individual drive So I decided to use the //screen// command and see if I could make that work.
+<code bash wipedrives2.sh>
+#! /usr/bin/env bash
+for drive in a b c d e f g h
+do
+   screen -dmS sd$drive bash -c "dd if=/dev/zero | pv -petrs 580G | dd of=/dev/sd$drive"
+done
+</code>
+Basically, we're using a bash for loop to grab all the drive names (I just used the last letter), running screen and immediately detaching the new process after telling it to run //bash -c// and the command after it in quotes (so it would not interpret the pipes in our current, non-screen shell). I'm running this right now, and //pv// is predicting it will be done in 11.5 hours, or less than a third of the time. BUT, it is really heating up the office with 7 drives being continuously written to at the same time.
+**Warning**: When SystemRescueCD boots, it tries to assemble any mdadm (software RAID) arrays, and since they are locked, //dd// and //wipefs// won't be able to write to them (maybe). In that case, do the following:
+<code bash>
+# find any mdadm volumes running on Linux
+cat /proc/mdstat
+# assuming it showed you md127 was running (normal)
+mdadm --stop /dev/md127
+# it should stop the MD array and make the individual drives accessible
+</code>
 ==== Rename Server ====
@@ Line 17: / Line 84: @@
 <code bash>
 # change the host name, and the postfix name if that is installed
-sed -i.old 's/oldname/newname/g' /etc/hostname /etc/hosts /etc/mailname /etc/postfix/main.cf
+sed -i.old 's/oldname/newname/g' \
+   /etc/hostname \
+   /etc/hosts \
+   /etc/mailname \
+   /etc/postfix/main.cf \
+   /etc/camp/sysinfo-client/sysinfo-client.yaml \
+   /etc/msmtprc
 /etc/init.d/hostname.sh start
 # update the aliases, if they exist
@@ Line 164: / Line 237: @@
 ===== Shell (mainly BASH) =====
+==== Here Documents ====
+Most unix users are familiar with echo'ing something to STDOUT so it can be used as STDIN for a following script (using the pipe). However, this is usually limited to a single line.
+A **here document** is a way of having multiple lines processed at one time. In many cases, you can have similar functionality using quotes, but here documents are more robust.
+For example, a simple test of a newly built mail system might include creating a file with all of the headers necessary, then passing that to //sendmail// for processing. However, you can skip a step and simply send the message directly using a here document.
+<code bash>
+sendmail user@example.com << EOF
+To: user@example.com
+from: root@example.org
+Subject: test
+This is a test
+EOF
+</code>
+The entire block above is one command. Here is the breakdown.
+  - //sendmail user@example.com// is a standard sendmail invocation which assumes the message itself is coming from STDIN.
+  - //<<// marks the beginning of a //here document//
+  - //EOF// is the tag which will mark the end of the text for the here document
+  - Everything up to the EOF is the actual string to be passed to sendmail
+  - //EOF// at the end marks the end of the here document. **Note**: there must be no leading or trailing whitespace. The tag must be exactly as entered after the << (case sensitive), and must be the only thing on the final line.
+This only touches the surface of here documents. See [[https://tldp.org/LDP/abs/html/here-docs.html]] for a lot more information.
 ==== Find files within date range containing text ====
@@ Line 173: / Line 273: @@
 This is very fast, since the find command rapidly decreases the number of messages which must be scanned (he has almost 300k e-mails in various folders, and it took less than 2 seconds).
+==== Find newest files in a directory tree ====
+This will go through an entire directory tree under the current directory and locate the newest 5 files.
+<code bash>
+find . -type f -exec stat --format '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head
+</code>
+  * Change //find .// to //find /some/path// to change the starting directory
+  * Change //head// to //head -n 10// to grab the newest 10 files.
+  * You can add any kind of filter also, so entering //-iname '*.jpg'// after the //-type f// would only find files ending in jpg.
 ==== Count all files in directory tree(s) ====
@@ Line 323: / Line 436: @@
   * https://linuxize.com/post/how-to-add-swap-space-on-debian-9/
   * https://docs.nextcloud.com/server/18/user_manual/files/access_webdav.html
+  * https://www.cyberciti.biz/faq/create-a-freebsd-swap-file/
+  * https://www.mybluelinux.com/test-imap-with-telnet/
+  * https://serverfault.com/questions/131627/how-to-inspect-remote-smtp-servers-tls-certificate#131628]