Tech blog of WEN Yang

An incomplete list of not-that-frequently-used but handy tools in Linux

Lots of programs in the Unix/Linux tend to follow the philosophy of "do one thing, do it well", and they are often designed to work together to do complicated jobs. Therefore, if we know what tools are available out there, we can break down the tasks into a few steps and use the most suitable tool for each of them. Usually, finding a right tool for the work in hand might well worth the time, if (that's a big if) you can easily find one and learn to use it quickly.

Some tools we use almost everyday: ls, cd, cp, mv, rm, pwd, find, grep, alias, locate, history, ... I am familiar with their common usage, at least for the basic ones. For those advanced cases, I know where to look for help quickly.

Of course there are also powerful tools in my arsenal, such as Sed, Awk, and Emacs. I can do a lot of things with them. But there is still times when some of the following (small) tools come in handy. Unfortunately, since they are not used as often, I tend to forget their usage and need to dig into the details of manual pages, which is less desirable. So I'll try to give some typical examples of how they can be used.

cut - remove sections from each line of files
paste - merge lines of files
sort - sort lines of text files
tr - translate or delete characters
tac - concatenate and print files in reverse
wc - display a count of lines, words  and  characters  in  a file
uniq - report or filter out repeated lines in a file
......

I'll put the details in my wiki. The directly link is here.

Man (Manual) -> PDF

groff -Tps -m man filename.n | ps2pdf - output.pdf

or

groff -Tps -m man filename.n > output.ps
ps2pdf output.ps

Using SED and AWK to find total demand from an origin for a certain time interval

Example1:

# sed -n '/21600 1 4/, /^}/ {/^{/p;}' demand.dat > test1

# awk '$2 == "11220" { x += $4 } END { print "total flow per interval: " x "\ntotal flow per hour: " x*4}' test1

Note:

• In the first line, 21600 is the time stamp, 1 is the type, 4 is the scale.
• In the second line, 11220 is the Node ID.

Example2:

# awk -f count_orig.awk orig=11220 Demand/demand_newpath_Xhat_0600.dat

Use script count_orig.awk, whose contain is the following:

$2 == orig { x += $4; print $2 "\t" $3 "\t" $4 }

END { print "total flow per interval: " x "\ntotal flow per hour: " x*4}

Note: "Demand/demand_newpath_Xhat_0600.dat" is the demand file for 6am.

Run matlab in batch mode

Use option "-r MATLAB_command" to run matlab in batch mode.

Automatically run the specified MATLAB M-file, either commands supplied with MATLAB or your own M-files, immediately after MATLAB starts. This is also referred to as calling MATLAB in batch mode. Separate multiple commands with commas or semicolons (;). M-files must be on the MATLAB path or in the MATLAB startup directory. Do not include the pathname or a file extension.

Example: matlab -nojvm -nosplash -r MyCommand

Reference: MATLAB Documentation - Desktop Tools and Development Environment - Startup Options

We can now use "system" (need to include < cstdlib >) to call a shell-script, which in turn calls matlab to do something.

eject, screen, lftp ...

eject - eject removable media

When you umount /mnt/usb, sometimes the LED is still on, then you can try this

eject /mnt/usb

screen - screen manager with VT100/ANSI terminal emulation

SYNOPSIS
       screen [ -options ] [ cmd [ args ] ]
       screen -r [[pid.]tty[.host]]
       screen -r sessionowner/[[pid.]tty[.host]]

lftp -

To be continued ...

[Script] run DynaMIT, check runtime

#!/bin/bash
echo '=== Running CorbaFreeDynaMIT from current directory ==='
STARTTime=`date`
echo 'Preparing to delete __* files in current directory and all sub directory'
ls __e*
if [ $? = 0 ]
then 
    echo -n "Really want to remove these files? (y/n):"
    read yesORno
    if [ $yesORno = y -o $yesORno = Y ]
        then
        echo 'find . -maxdepth 1 -name "__*" -exec rm -v {} \;'
        find . -maxdepth 1 -name "__e*" -exec rm -v {} \; && echo "Files removed."
    else
        echo "You did not choose 'y'. Continue without deleting"
    fi
fi
time ~/CorbaFreeDynaMIT/DynaMIT/DynaMIT dtaparam.dat
echo 'Script' $0 'started at: ' $STARTTime
echo -n 'Script' $0 'finished at: '&& date
echo 'Done.'

nohup - run a command immune to hangups, with output to a non-tty

In bash, use

nohup ~/CorbaFreeDynaMIT/DynaMIT/DynaMIT dtaparam.dat 1>log_stdout 2>log_stderr &