I love shell productivity hacks. I believe that if you work in a Linux environment, you owe it to yourself to become as good as you can be at working with your shell of choice (e.g. bash). You see, most people who have worked with Linux for any length of time have some level of skill with the shell, that level is usually mediocre at best. But, you do meet the occasional person who wows you with what they can accomplish and I will tell you right now that their skill does not come from superior knowledge (or at least not fully from superior knowledge). It is all about maximizing the effect of the knowledge you do have, finding little tricks and hacks that will save you, potentially less than a second, every time you perform a particular action. Thing is, some actions you might do hundreds of times per day, so those seconds really start to add up, especially once you have accumulated dozens of these hacks. Anyway, here is one such tip.
I find that whenever I get a large data file from somewhere (i.e. extract some data from a database, crawl some sites and dump the data in a file) I always need to do just that little bit of extra processing before I can actually use it. This processing is always just non-trivial enough and I do it just uncommonly enough for me to always forget exactly how to go about it. Of course, this is to be expected, if you learn something and want it to stick you have to keep doing it. It's all part and parcel of how our brain works when it comes to learning new skills, but that doesn't make it any less annoying.
Back to our data file, for me I find that I almost always need to do 3 things (amongst others) before doing anything else with my file.
- delete the first line (especially when pulling data out of the database)
- delete the last line
- remove all blank lines
Don't ask me why but for whatever reason, you always get an extraneous first line and unexpected blank lines (and less often an extraneous last line) no matter how you produce the file :).
It may or may not surprise you to know that the bash shell has a very rich array of convenient shortcuts that can make your life, working with the command line, a whole lot easier. This ability to edit the command line using shortcuts is provided by the GNU Readline library. This library is used by many other *nix application besides bash, so learning some of these shortcuts will not only allow you to zip around bash commands with absurd ease :), but can also make you more proficient in using a variety of other *nix applications that use Readline. I don’t want to get into Readline too deeply so I’ll just mention one more thing. By default Readline uses emacs key bindings, although it can be configured to use the vi editing mode, I however prefer to learn the default behavior of most applications (I find it makes my life easier not having to constantly customize stuff). If you’re familiar with emacs then many of these shortcuts will not be new to you, so these are mostly for the rest of us :).
Command Editing Shortcuts
- Ctrl + a – go to the start of the command line
I am not a Linux newbie, but neither am I an expert. So, I sometimes tend to come across some really cool stuff and I just have to share.
Here is the story. We use this application at work (I won’t point any fingers :)) which tends to run as a whole lot of processes when it starts up, and these processes don’t all have the same parent process either. Once in a while this application can get into a bad state where you have to manually kill all the processes to restart everything. Now as you might imagine this can be a bit of a pain since there are quite a few processes even if you only count the parents.
Here is where some bash magic comes to the rescue. Luckily for us, this application runs as a separate Linux user so all the processes have the same user id and we’re able to use the following command to kill everything in one fell swoop.
ps -ef | grep ^1002 | cut -c 5-15 | sudo xargs kill
Here is a quick rundown of what it does:
- the ps command is obvious, we get the processes that are currently active