Shell Scripting (Media Engineering Architecture & Deployment)

# Shell Scripting

Learn the basics of shell scripting with Bash.

This material is part of [architecture & deployment course](https://github.com/MediaComem/comem-archidep) for [Media Engineering](https://heig-vd.ch/formations/bachelor/filieres/ingenierie-des-medias).

**You will need**

* A Unix CLI

**Recommended reading**

* [Command Line Introduction](../cli/)

---
## What is a script?

.breadcrumbs[<a href="#1">Shell Scripting</a>]

In a Unix-like operating system, a file that can be executed should be one of
the following:

* A **binary file**, which contains machine-readable binary code that has been
  compiled from source code.
* A **script**, which is a file containing code that is dynamically interpreted.

---
### How is a script executed?

.breadcrumbs[<a href="#1">Shell Scripting</a> > <a href="#2">What is a script?</a>]

When an executable text file is run, a Unix-like operating system looks for a
**shebang** on the first line. A shebang is a line with the following format:

```
#!`interpreter` optional-args
```

For example, the following is a valid shebang:

```
#!/bin/bash
```

In this example, it tells the operating system that the **interpreter** which
should run this file is `/bin/bash`, meaning that this is a [Bash][bash] script.

> Note that there must not be any space between `#!` and the path of the
> interpreter `/bin/bash`.

---
### What can I put in a script?

.breadcrumbs[<a href="#5">!/bin/bash</a>]

In a bash script, you can put anything you could type in a Bash shell:

```bash
#!/bin/bash
echo Hello World
```

In a PHP script, you can put any PHP code you want:

```php
#!/usr/bin/php
<?php
echo 'Hello World';
?>
```

Basically, what you can put in a script depends on the interpreter you're using.

---
### How do I create a script?

.breadcrumbs[<a href="#8">!/usr/bin/php</a>]

Simply create your script:

```bash
$> printf '#!/bin/bash\necho Hello World' > test.sh
```

> The `printf` (**print** **f**ormat) command is similar to the `echo` command
> but it has better support for special characters like new lines (`\n`).

Make it executable:

```bash
$> chmod +x test.sh
```

And run it:

```bash
$> ./test.sh
Hello World
```

---
### All kinds of scripts

.breadcrumbs[<a href="#8">!/usr/bin/php</a>]

The following are a few examples of shebangs, but it is nowhere near exhaustive:

> Of course, the path to the interpreter must correspond to the actual path of
> the command used (`sh`, `bash`, `php`, etc). It might differ on your machine.
> Use `which bash` to find the location of the Bash executable, for example.

---
## What is shell scripting?

.breadcrumbs[<a href="#8">!/usr/bin/php</a>]

**Shell scripting** is the practice of writing scripts that contain series of
shell commands that you want to be able to reuse.

Any script with a shell as the interpreter is a "shell script".

> A script using PHP as the interpreter is still a script, but it's not a "shell
> script". It's a PHP script.

---
class: center, middle
## Shell script basics

.breadcrumbs[<a href="#8">!/usr/bin/php</a>]

> A few pointers on writing Bash scripts (compatible with most POSIX shells).

---
### Commands

.breadcrumbs[<a href="#8">!/usr/bin/php</a> > <a href="#12">Shell script basics</a>]

You can use any shell command in a shell script:

```bash
#!/bin/bash
echo Hello World
date
ls
```

This script could print:

```
Hello World
Thu Jan 10 23:46:52 CET 2019
file.txt directory ...
```

---
### Working directory

.breadcrumbs[<a href="#14">!/bin/bash</a>]

By default, a script executes in the current shell directory.

You can use `cd` to move around to other directories:

```bash
#!/bin/bash
pwd
cd /home
pwd
```

This script could print:

```
/some/where/over/the/rainbow
/home
```

Assuming it was executed from the `/some/where/over/the/rainbow` directory.

---
### Variables

.breadcrumbs[<a href="#16">!/bin/bash</a>]

You can declare and reuse variables in scripts:

```bash
#!/bin/bash
FOO=bar
echo $FOO
```

If your variable contains whitespace (spaces, new lines, etc),
be sure to quote it when declaring and using it to avoid issues:

```bash
#!/bin/bash
FOO="bar baz"
echo "$FOO"
```

---
#### Store the output of commands

.breadcrumbs[<a href="#19">!/bin/bash</a>]

You can store the result of a command in a variable by wrapping it with backticks:

```bash
#!/bin/bash
FILES=\`ls -1`
NUMBER_OF_FILES=\`echo "$FILES" | wc -l`
echo There are $NUMBER_OF_FILES files
```

This script would output 10 if there are 10 files in the current directory.

---
#### Environment variables

.breadcrumbs[<a href="#21">!/bin/bash</a>]

Environment variables are also available as variables in shell scripts:

```bash
#!/bin/bash
echo $PATH
```

To set an environment variable, do it like you would in any Bash shell:

```bash
#!/bin/bash
export FOO=bar
```

> Of course, the `$FOO` environment variable in this example will only be set in the context of this script and its child processes (if any).

---
### Conditionals

.breadcrumbs[<a href="#24">!/bin/bash</a>]

Bash has a classic `if/then/else` construct:

```bash
#!/bin/bash
FOO="bar"

if [[ "$FOO" -eq "foo" ]]; then
  echo FOO is foo
elif [[ "$FOO" -eq "bar" ]]; then
  echo FOO is bar
else
  echo foo is something else
fi
```

> The `[[  ]]` syntax is a Bash [test construct][bash-test-constructs].
> Also see Bash [other comparison operators][bash-comparison-operators].

---
#### The `test` built-in command

.breadcrumbs[<a href="#26">!/bin/bash</a>]

The `test` command which comes with Bash is another way to write some conditions:

```bash
#!/bin/bash

EMPTY_VAR=
FULL_VAR="full"
FILE="/path/to/some/file"

if test -z "$EMPTY_VAR"; then
  echo variable is empty
fi

if test -n "$FULL_VAR"; then
  echo variable is not empty
fi

if test -f "$FILE"; then
  echo file exists
else
  echo file does not exist
fi
```

> See Bash [file test operators][bash-file-test-operators] and [other comparison operators][bash-comparison-operators].

---
### Loops

.breadcrumbs[<a href="#28">!/bin/bash</a>]

Bash has a `for` loop:

```bash
for item in one two three; do
  echo $item
done
```

The above code would print:

```
one
two
three
```

> Bash also has `while` and `until`.
> See [loops & branches][bash-loops].

---
### Special variables

.breadcrumbs[<a href="#28">!/bin/bash</a>]

Bash has a number of [special variables][bash-special-vars] which are always available:

For example, this script says hello to the name passed as the first argument:

```bash
#!/bin/bash
echo Hello $1
```

---
### The `set` built-in command

.breadcrumbs[<a href="#31">!/bin/bash</a>]

The `set` command is specific to Bash and can be used to toggle its [option
flags][bash-option-flags].

For example, the `-e` option aborts the script if an error occurs, while the
`-x` option prints commands before executing them:

```bash
#!/bin/bash
set -ex
echo Hello World
cat file-that-does-not-exist
echo Done
```

This script could print:

```
+ echo Hello World
Hello World
+ cat file-that-does-not-exist
cat: file-that-does-not-exist: No such file or directory
```

> Note that each command is printed with a leading `+` before being executed,
> and that the script stops as soon as an error occurs
> (which is **not the case by default**).

---
### Functions

.breadcrumbs[<a href="#33">!/bin/bash</a>]

You can isolate pieces of code in a function. The special argument variables
`$1`, `$2`, etc represent the arguments to the function:

```bash
#!/bin/bash

print_hello() {
  echo Hello $1
}

print_hello World
```

This script would print `Hello World`.

---
#### Variable scope

.breadcrumbs[<a href="#35">!/bin/bash</a>]

Note that normal Bash variables have no scope, i.e. they are available in the
whole file and every function.

To declare a variable that is local to a function, use the `local` keyword:

```bash
#!/bin/bash

print_hello() {
  local name=$1
  echo Hello $name
}

print_hello World
echo $name
```

This script would print `Hello World` and an empty line,
since `$name` is only defined within the `print_hello` function.

---
## References

.breadcrumbs[<a href="#37">!/bin/bash</a>]

* [Shell Style Guide](https://google.github.io/styleguide/shellguide.html)
* [Advanced Bash Scripting Guide](https://www.tldp.org/LDP/abs/html/)
  * [Test Constructs][bash-test-constructs]
  * [File Test Operators][bash-file-test-operators]
  * [Other Comparison Operators][bash-comparison-operators]
  * [Loops & Branches][bash-loops]
  * [Local Variables][bash-locals]
  * [Special Shell Variables][bash-special-vars]
  * [Starting Off With a Sha-Bang][bash-shebang]
* [Shell Script][shell-script]

[bash]: https://en.wikipedia.org/wiki/Bash_(Unix_shell)
[bash-comparison-operators]: https://www.tldp.org/LDP/abs/html/comparison-ops.html
[bash-file-test-operators]: https://www.tldp.org/LDP/abs/html/fto.html
[bash-locals]: https://www.tldp.org/LDP/abs/html/localvar.html
[bash-loops]: https://www.tldp.org/LDP/abs/html/loops.html
[bash-option-flags]: https://www.tldp.org/LDP/abs/html/options.html#OPTIONSREF
[bash-shebang]: https://tldp.org/LDP/abs/html/sha-bang.html
[bash-special-vars]: https://tldp.org/LDP/abs/html/refcards.html#AEN22402
[bash-test-constructs]: https://www.tldp.org/LDP/abs/html/testconstructs.html
[node]: https://nodejs.org
[php]: http://php.net
[python]: https://www.python.org
[ruby]: https://www.ruby-lang.org
[sh]: https://en.wikipedia.org/wiki/Bourne_shell
[shell-script]: https://en.wikipedia.org/wiki/Shell_script
[zsh]: https://en.wikipedia.org/wiki/Z_shell