Avoiding Bash frustration — Use Python for shell scripting

I never got used to Bash programming syntax. Whenever I have to write a more-than-trivial bash script, the strange syntax annoys me, and I have to Google every little thing I need to do, starting from how to do comparisons in if statements, how to use sed , etc.

For me, using Python as a shell scripting language seems like a better choice.
Python is a more expressive language. It is relatively concise. It has a massive built-in library that let you perform many tasks without even using shell commands, it is cross-platform and it is preinstalled or easily installed in many OS’s.
I am aware that some other dynamic languages (e.g Perl, Lua) might also be very suitable for shell programming, but I (and my team) work with Python daily and familiar with it, and it gets the jobe done.

In my last project, after some bash frustration, I decided to refactor a bloated set of bash scripts with a CLI-style Python script. A side-result of this work is a small, single helper file, which I am going to share with you here, with few utilities that bridge the gap to easily let you use Python for shell scripting.

In an effort to make this utility runnable ubiquitously, I made it compatible with both Python 2.7 and Python 3.5+ (Python 2.7 is preinstalled in Ubuntu since version 14). There are no 3rd-party requirements — only Python’s standard library is used, so no pip install is required.
I tested it on Mac and Ubuntu.

For shell programming, we need to be able to execute shell commands conveniently. The most extensive Python function for that is subprocess.Popen() . However, Popen() might feel too raw to use easily. It also has some compatibility changes between Python 2/3 and some missing features in Python2.

The core function I provide here is sh() , which is a friendly wrapper around subprocess.Popen(). It let you execute a shell call from Python, deciding whether to:

• Capture stdout/stderr to a string or print it.
• Time-out the call after x seconds
• Terminate the calling Python script on failure of the shell call.
• Terminate the calling Python script on timeout of the shell call.
• Echo the command string of the shell call.
• Run the command inside a shell [ like subprocess.Popen(shell=true) ]. This is considered an insecure practice due to the possibility of a shell injection, but allows many convenient features in shell calls, like pipes (|), environment variables interpolation, executing multiple statements with && or ; at once call and more, so if your script gets no user input, or you trust your input, you may opt to use it.
• Apply Pythonic formatting arguments to the shell command before executing it.
• And more …

Other utilities let you:

• Log/print to stdout with ANSI colors according to the logging level.
• Get user input prompt from stdin, with compatibility to both Python 2.7 and Python 3+.

An example script which:

• Asks the user whether to pull for a new docker image
• Removes the running container for this image (if any)
• Runs a new container for the image.
• Outputs the first 5 seconds of the new container log.

Output:

A caveat I discovered with using Python for shell scripting, is that child processes are not terminated when the parent Python process dies. A solution I found for that, which is embedded in the utility, is using the exit hook [ at_exit() ] to kill any child processes that are not terminated yet. This approach will work for soft kills like Ctrl-C, but not for a more aggressive kill like when your python script is terminated using kill -9 , and may leave the child shell command running. I am open to new ideas on how to work-around this drawback. However since most shell scripts are executing short-lived commands, I don't see this as a showstopper.

How to consume:

Just copy the peasyshell.py file near your Python shell script, and import it.

• Another usage sample
• GitHub repo

The code is distributed under the Apache v2 OSS license.

Tip: You can make your Python script executable:

1. Add a shebang line at the top of your script:

#!/usr/bin/env python2
from peasyshell import *
...

2. Make your script executable:

chmod +x my_app.py

3. Run:

./my_app.py

Have fun scripting.