File sizes as integers in bash using ls

17 de Junho de 2010, por Desconhecido - 0sem comentários ainda

How to get properly working logical tests in bash that use the output of ls to get a file size?

I've struggled with this for a while, so now that I (believe) I solved it, I've decided to share this info.

The first thing is to get the file size field from ls output. Obviously, you'd need the -l option. From the ls man page:

       -l     use a long listing format

For instance:

$ ls -l one
-rw-r--r-- 1 rborges users 40K 2010-06-18 19:20 one

So we would like to get the 5th field from this output. I use the cut program for this, defining the field separator as the space with the -d option and the 5th field with the -f option. From the cut man page:

       -d, --delimiter=DELIM
              use DELIM instead of TAB for field delimiter

       -f, --fields=LIST
              select only these fields;  also print any line that contains no delimiter character, unless the -s option is specified

Now our example becomes:

$ ls -l one | cut -d" " -f 5
40K

Notice we have an extra line in the output (also, the commands are piped one into the other, but I won't explain pipes here).

The reason there is an extra line, is that in my system, ls is configured to display listings with color.

Also, there is a "K" after the size, indicating this particular file has 40 Kilobytes. This is because in my system ls is configured to print file sizes in human-readable form. (694M is much more readable than 726896644, right?)

Both the human-readable and the color options may be set session-wide, and you could just reset them. But then your script might not be as portable as possible. SoI have found out how to disable both options on the command line, and this is what I want to share.

The importance of this is to get the file size in a variable as an integer. This, in its turn, allows bash to perform logical tests with the integer. If you don't properly command ls, you may end up stuck trying to get a logical test to work, and although it seems perfect, it isn't, because your variable isn't holding an integer - it is holding a string with ANSI codes, or a string with a K or M in its end.

To verify this, check the difference:

$ TEST=`ls --color=always -l four | cut -d" " -f 5` ; echo $TEST; if [ $TEST -eq 10 ]; then \

echo "TEST equals 10"; else echo "TEST does not equal 10"; fi
10
bash: [: too many arguments
TEST does not equal 10
$ TEST=`ls --color=never -l four | cut -d" " -f 5` ; echo $TEST; if [ $TEST -eq 10 ]; then \

echo "TEST equals 10"; else echo "TEST does not equal 10"; fi
10
TEST equals 10

Notice the if error (the "too many arguments line"). This occurs because you have something that is not an integer being used in an integer-only test. Also, the test fails although the file size do is 10.

So you have to make sure you disable your ls colors with the --color=never option. From the ls man page:

 --color[=WHEN]
              control whether color is used to distinguish file types.  WHEN may be `never', `always', or `auto'

Now the other caveat: you may have your ls set to print human-readable sizes.

For instance:

$ for i in {1..10000}; do echo "bla" >> one; done
$ ls -l one                                      
-rw-r--r-- 1 rborges users 40K 2010-06-18 19:20 one
$ ls --color=never -l one | cut -d" " -f 5
40K

So if your ls is set to print in human-readable format, you may get the file size in Kilobytes or Megabytes.

The solution to this is to use the --block-size option. From the ls man page:

 --block-size=SIZE
              use SIZE-byte blocks

$ ls -l --block-size=1 one                
-rw-r--r-- 1 rborges users 40000 2010-06-18 19:20 one

Putting both things together:

$ ls --color=never --block-size=1 -l one | cut -d" " -f 5
40000

So now the output from your ls is sctrictly without color, unit and properly trimmed. You can now use this script to get file sizes and use the size in more elaborate scripts. Just to be easier, you might want to define a function for this:

function file_size {

    return `ls --color=never --block-size=1 -l $1 | cut -d" " -f 5`

}

And a script to remove all the 0-sized files from the current path would be:

FILE_LIST=`ls --color=never -1`

for FILE in $FILE_LIST; do

    file_size $FILE

    if [ $? -eq 0 ]; then

        rm $FILE

    fi

done


Enjoy! But remember to backup you files before trying this script for yourself! (I give no warranties! :)