SED Guide
--------------
###FEATURES COMMON TO BE BOTH AWK & SED ####
1. Both are scripting languages
2. Both work primarily with text files
3. Both are programmale editors
4. Both accept command-line optinos and can be scripted (-f script_name)
5. Both GNU versions support POSIX (GREP) and EGREP RegExes
6. Lineage = ed (editor) -> sed -> awk
###SED'S FEATURES###
1. Non-interactive editor
2. Stream Editor
a. Manipulates input - performing edits as instructed
b. Sed accepts input on/from: STDIN (Keyboard), File, Pipe (|)
3. Sed Loops through ALL input lines of input stream or file, by DEFAULT
4. Does NOT operate on the source file, by default. (Will not clobber the orginal file, unless instructed to do so)
5. Supports addresses to indicate which lines to operate on: /^$/d - delete blank lines
6. Stores active (current) line the 'pattern space' and maintains a 'hold space' for usage
7. Used primarily to perform search and replcaes
###AWK's FEATURES###
1. Field processor
2. Used for reporting (extracting specific columns) from data feed
3. Supports programming constructs
a. loops (for,while,do)
b. conditions (if,then,else)
c. arrays (lists)
d. function (string, numeric, user-defined)
4. Automatically tokenizes wors in a line for later usage - $1, $2, $3, etc. (this is based on the current delimeter)
5. Automatically loops through input like Sed, making lines availale for processing
6. Ability to execute shell commands using 'system()' functions
####REGULAR EXPRESSIONS (RegEx) REVIEW###
Regular Expressions (RegExes) are key to mastering Awk & Sed
###METACHARACTERS###
^ - carrot symbal - Matches the character(s) beginning of a line
$ - Mathes the charater(s) at end of the line
Task: Match line which contailns only 'dog':
a. sed -ne '/^dog/p' animal.txt
b. sed -ne '/^dog$/p' animal.txt
c. sed -ne '/^dog$/p' - Reads from STDIN. Press enter after each line. Terminate with CTRL-D.
d. cat animails.txt | sed -ne '/^dog$/Ip' - print matches case-insensitively
. - matches any charecter (typically except new line)
a. sed -ne '^d...$/Ip' ani.txt
b. sed -ne '^d.../Ip' ani.txt
####REGEX QUANTIFIERS#####
* - 0 or more matches of the previous charecter
+ - 1 or more matches of the previous charecter
? = 0 or 1 of the preveious charecter
b. sed -ne '/^d.\+/Ip' ani.txt
Note: Escape quantifiers in RegExes using the escape character '\'
###CHARACTERS CHASSES###
Allows to search for range of characters
a. [0-9]
b. [a-z][A-Z]
a. sed -ne '/^d.\+[0-9]/Ip' animal.txt
####INTRO TO SED####
Usage:
1. sed [optiions] 'instruction' file | PIPE | STDIN
2. sed -e 'instruction' -e 'instruction2' ...
3. sed -f script_file_name file
Note: Execute sed by indication instrution on one of the following:
1. Command-line
2. Script File
Note: sed accepts instructions base on '/patten_to_math/'
###Print Specific Lines of file###
Note '-e' is optional if there is only 1 instuction to execute
sed -ne '1p' ani.txt - prints first line of file
sed -ne '2p' ani.txt - prints second line of ile
sed -ne '$p' ani.txt - prints last printable line of file
sed -ne '2,4p' ani.txt - prints lines 2-4 from file
sed -ne '1!p' ani.txt - prints ALL lines EXCEPT line #1
sed -ne '1p;4p' ani.txt - prints 1st and 4th line of file
sed -ne '/dog/p' ani.txt - prints ALL lines containing 'dog' - case-sensitive
sed -ne '/dog/Ip' ani.txt - prints ALL lines containing 'dog' - case-insensitive
sed -ne '/[0-9]/p' ani.txt - prints ALL linues iwth AT LEASt 1 numeric
sed -ne '/cat/,/deer/p' ani.txt - print ALL lines begininning with 'cat', ending with 'deer'
sed -ne '/deer/,+2p' ani.txt - prints the line with 'deer' plus 2 extra lines
###Delete Lines using Sed Addresses###
sed -e '/^$/d' ani.txt - deletes blank lines from file
Note: Drop '-n' to see the new output when deleting
sed -e '1d' ani.txt - deletes the first line from ani.txt
sed -e '1~2d' ani.txt - deletes every 2nd line beginning with line 1 - 1,3,5,...
#####Save Sed's changes using Outpu Redirection####
sed -e '/^$/d' ani.txt > ani2.txt - deletes blank lines from file and create new output file 'ani2.txt'
####SEARCH & REPLACE USING Sed####
General Usage:
sed -e 's/find/replace/g' ani.txt - replaces 'find' with 'replace'
Note: Left Hand Side (LHS) supports literals and RegExes
Note: Ride Hand Side (RHS) supports literals and back references
Examples:
sed -e 's/Centos/Redhat/' - replaces 'Centos' to 'Redhat' on STDIN to STDOUT
sed -e 's/Centos/Redhat/I' - replaces 'Centos' to 'Redhat' on STDIN to STDOUT (Case-Insensitive)
Note: Replacements occur on the FIRST match, unless 'g' is appended to the /s/find/replace/g sequence
sed -e 's/Centos/Redhat/Ig' - replaces 'Centos' to 'Redhat' on STDIN to STDOUT (Case-Insensitive & Global)
Task:
1. Remove ALL Blank Lines
2. Substitute 'cat', regradless of case, with 'Tiger'
Note: Whenver using '-n' opetion, you MUST specify the print modifier 'p'
sed -ne '/^$/d' -e 's/cat/Tiger/Igp' ani.txt - Removes Balnk lines & substitutes 'cat' with 'Tiger'.
OR sed -ne '/^$/d; s/cat/Tiger/Igp' ani.txt - Does the same as above
####Update Sources file - Backup Source File####
sed -i.bak -e '/^$/d; s/cat/Tiger/Igp' ani.txt -Performs as above, but ALSO replaces the source file
#####Search & Replace (Text Substitution) Continues####
sed -e '/address/s/find/replace/g' file
sed -e '/Tiger/s/dog/mutt/g' ani.txt
sed -ne '/Tiger/s/dog/mutt/gp' ani.txt - substitues 'dog' with 'mutt' where line contains 'Tiger.
sed -ne '/Tiger/s/dog/mutt/Ip' ani.txt
sed -ne '/^Tiger/s/dog/mutt/gIp' ani.txt - Updates lines that begin with 'Tiger'
sed -ne '/^Tiger/Is/dog/mutt/gIp' ani.txt - Updates lines that begin with 'Tiger' (Case-Insensitive)
#####Focus on the Right Hand Side (RHS) of search and Replace Functions in SED####
Note SED reserves a few charecter to help with substitutions base on the mathed pattern from the LHS
& = The full value of LHS (Pattern Matched) OR the values in the pattern space
Task:
Intersperse each line with the work 'Animal '
sed -ne 's/.*/&/ ani.txt - replaces the mathed parrern with the matched pattern
sed -ne 's/.*/Animal &/p' animals.txt - Intersperses 'Animal' on each line
sed -ne 's/.*/Animal &/p' animals.txt - Intersperses 'Animal:' on each line
sed -ne 's/.*[0-9]/&/p' animals.txt - returns animals with at least 1 numberic at the end of the name
sed -ne 's/[a-z][0-9]\{4\}$/&/Ip' animals.txt - returns animal(s) with 4 numeric values at the end of the name
sed -ne 's/[a-z][0-9]\{1,4\}$/&/Ip' animals.txt - returns animal(s) with at least 1, up to 4 numeric values at the end of the name
####Grouping & Backreferences####
Note: Segment mathes into backreferences using escaped parenthesis: \(RegEx\)
sed -ne 's/\(.*\)\([0-9]\)/&/p' animals.txt - This create 2 variables: \1 & \2
sed -ne 's/\(.*\)\([0-9]\)/\1/p' animals.txt - This create 2 variables: \1 & \2 but references \1
sed -ne 's/\(.*\)\([0-9]\)/\2/p' animals.txt - This create 2 variables: \1 & \2 but references \2
sed -ne 's/\(.*\)\([0-9]\)/\1 \2/p' animals.txt - This creates 2 variables: \1 & \2 but references \1 & \2
####Apply Changes to Multiple Files####
Sed supports Globbing
sed -ne 's/\(.*\)\([0-9]\)$/\1 \2/p' animals*txt
###Sed Scripts###
Note: sed supports scripting, which mean, the ability to dump 1 or more instructions into 1 file
sed -f script_file_name text_file
sed -f animals.sed animals.txt
Task:
Perform multiple transformations
1. /^$/d - Remove blank lines
2. s/dog/frog/I - substitues globally, 'dog' with 'frog' - (case-insensitive)
3. s/tiger/lion/g - substitues globally, 'tiger' with 'lion' - (case-insensitive)
4. s/.*/Animals: &/ - Interspersed 'Animals:'
5. s/animals/mammals/Ig - Replaced 'animals' with 'mammals'
6. s/\([a-z]\)*\([0-9]*\)/I\1/p - Strips trailing numeric values from alphas
Sed Scripting Rules:
1. Sed applies ALL rules to each lines
2. Sed applies ALL changes dynamically to the pattern space
3. Sed ALWAYS works with the current lines
Realtime Examples,
sed '/iamrbam/s/^/# /' - Search word iamrbam and append # at start of line
sed '/iamrbam/s/$/ #/' - Search word iamrbam and append # at end of line
sed -i -e '/iamrbam/s/^/# /'
