My name is Neo!

"UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity."
(Dennis Ritchie).

LinuxBanner

Linux Commands

Predefined and enviromental, variables


#$PATH, $USER, $HOME, $PS1, $HOSTNAME, $SHELL, $SSH_CLIENT, $SSH_CONNECTION #!/bin/bash echo $PATH #it reports the location of executables /usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games echo $USER #the user shady echo $HOME #the home directory of the user /home/shady echo $PS1 #it shows the prompt of command \[\e]0;\[email protected]\h: \w\a\]${debian_chroot:+($debian_chroot)}\[email protected]\h:\w\$ echo $HOSTNAME #it shows the name of your computer obideb echo $SHELL /bin/bash

Linux shell is "case-senstive". ls -al is not the same of ls -AL
Commands can be called with arguments (or flag) and parameters. Every command got a "manual", man ls
shows the manual of the command ls with all flag.

In order to start our course, let's download this file

Download Files

had you already clicked on the red button as every common user?
I would bet on it

wget Command

Right click on the link, copy link location, so come back to you our wonderful shell


wget http://www.spinfo.it/bioinflesson2/files/script.tar.gz

Extract Gzip files


tar zxvf script.tar.gz # z = Gzip type # x = extract # v = verbose (shows on standard output the files, while extraction) # f = use archive file or device ARCHIVE

Tar, The GNU version of the tar archiving utility, saves every information about files. Is Open-Source and with Gzip is a wonderful compression method.

Now run (execute) the script we've downloaded before.
Before, we need to set permissions for this file in order to run (EXECUTE) it as user.
chmod 744 creaFiles.sh
then run it ./creaFiles.sh


# GZIP -9 is the best compression method for Gzip. Best compression = More time to compress and extract. GZIP=-9 tar zcf namefile.tar.gz ./obiDir


Cat

cat is used to

concatenate files and redirect it to standard output.
Cat can concatenate more then one file, but is often used to redirect to standard output a file, maybe... with some "filter".


cat /etc/passwd #will print on monitor (standard output) the content of /etc/passwd file


Cut

cut - remove sections from each line of files


cut -d: -f1 /etc/passwd # -d to define a delimiter # -f to select the fields divided by delimiter. 1 means field 10

Let's show to standard output the file created by the script.


cat testo1.txt stefano,IT,unina marco,bio,unina serena,bio,unina mario,IT,igb

This is a "comma separated value" format. How to show only the 1st field?


cut -d"," -f1 testo1.txt stefano marco serena mario

And what if we'd like to show more then one field?


cut -d"," -f1,3 testo1.txt stefano,unina marco,unina serena,unina mario,igb

Is it possibile to use any other command to obtain the same result?


cat testo1.txt | awk -F, '{print $1 "," $3}' stefano,unina marco,unina serena,unina mario,igb

What is the best way? And what is the strange char "|" used?
Do you really think that the strongest stuff is the vertical line? Trust me, you do not know how much power is inside the letters "awk".
Just the acronym discover its power. AWK stay for:
Aho, Weinberger, but the last name is the most famous into the IT environment... Yes, modern computers exist, just because of a REAL GENIUS! Kernighan any reference to property or persons or "jobs" is purely coincidental...
awk is a scripting language, not just "a command", gawk is a newest implementation with a wonderful documentation. It is used in order to process text file structured as record and obtain reports. Syntax of AWK is similar to C, but it require more time to be explained as well.
gawk manual


An easy sample of use of AWK


cat testo1.txt | grep unina | grep IT | awk -F, '{print $1" esperto in "$2}'

Two different ways to obtain the same result... sometimes AWK looks like awful


cat /etc/passwd | awk 'BEGIN {FS=":"} ($7=="/bin/bash") {print $1}' root shady

Who said that without awk we cannot use bash?


cat /etc/passwd | grep /bin/bash | cut -d: -f1 root shady

Even if...sometimes it is really useful


awk -F',' '{print NF; exit}' testo1.txt #It will count the number of Column in a file, with field separator "," #e.g. with passwd file awk -F':' '{print NF; exit}' /etc/passwd 7

head and tail

Two really useful commands are, head and tail.
tail - output the last part of files head - output the first part of files.
Let's show the 1st and the last line of the file testo1.txt


head -1 testo1.txt && tail -1 testo1.txt ## && means AND stefano,IT,unina mario,IT,igb


sort and grep

sort - sort lines of text files
alfabetical sort is default


sort testo1.txt marco,bio,unina mario,IT,igb serena,bio,unina stefano,IT,unina


sort -r testo1.txt #reverse order stefano,IT,unina serena,bio,unina mario,IT,igb marco,bio,unina



# -n order by dimension, -k5 select the field of the command ls -l sent to the pipe ls -l | sort -k5 -n

grep, egrep, fgrep, rgrep - print lines matching a pattern if you want to filter a search or list, grep is what you need!
Let's filter our file testo1.txt, looking for the line that contain the word "marco"


grep marco testo1.txt marco,bio,unina


Now, the same way to obtain all line that contain the word "unina"


grep unina testo1.txt stefano,IT,unina marco,bio,unina serena,bio,unina


The -v flag, exclude result that match with the pattern

grep -v unina testo1.txt 
mario,IT,igb


The -c flag, count the line for the final result

grep -c -v unina testo1.txt 
1


Pipe and wc -l, will help when you cannot use a specific flag to count

grep -v unina testo1.txt | wc -l
1


What is the difference between...

grep 'a' testo1.txt | wc -l

grep 'a' -o testo1.txt | wc -l   # -o, --only-matching       show only the part of a line matching PATTERN


wc -w, print the word counts


grep unina testo1.txt | wc -w
3

## Another way
grep -c -P '(unina)' testo1.txt  ## -P?? What's that "P" stand for?
3


Pipe and regular expressions to improve filters.
[^m] means, "that starts with letter m"

grep -e "^m" testo1.txt marco,bio,unina mario,IT,igb


Pipe and regular expressions to improve filters.
-r stay for recursive -n for number of line -w stands match the whole word. -e to find a specific pattern with regex

grep -rnw "/home/shady" -e "pattern"


We will talk about Regular Expressions later...

Pipes

Pipes redirect OUTPUT of a command as INPUT for another one.
If we want to filter a list, or a file, with more then one option, pipes is the way.

Let's order a file, select the first 3 line and filter it with word "unina"


sort testo1.txt | head -3 | grep unina marco,bio,unina serena,bio,unina


Redirection to file

In order to change the redirection of a file from standard output to a file we can use > or >>
redirect standard output to a new file and if it doesn't exists, a new file will be created, but if it exists, the file will be overwritten. >> redirect standard output into an existing file, in order to "add", or better "append", new information into the file.


ls > list.txt #list will be redirected to a file called list.txt #no standard output will be shown


Let's "append" the current date to this file! Take care, do not destroy it.


echo $(date) >>list.txt