suggested improvements to these scripts

markp1989 · Apr 3, 2011

the other day I wrote this script, it downloads the latest .torrent files from the ubuntu web site

each unsuccessful download hits the Ubuntu webpage page 2 times, so i only run it every 6 hours (via cron) on the scheduled Ubuntu release day, I avoid running it too often as I don't want to over load their server.

Code:

#!/bin/bash
##markp1989
##script to download torrents of the latest ubuntu release. 


##setting the watch folder of your torrent client 
watch=/home/rtorrent/watch

##setting where to log any downloads 
log=/home/rtorrent/downloaded.ubuntuget

##cding to the watch folder  
cd $watch 


##scraping part of the releases page to find out the name of the latest version 
newest=$(curl http://releases.ubuntu.com/ | grep ")</a>" | cut -f2 -d'"' | cut -f1 -d'/' | tail -n 1)

##setting the base url  which is http://releases.ubuntu.com/NameOfNewestRelease 
baseurl="http://releases.ubuntu.com/$newest/"


        #running a loop for every torrent link on the latest release page

        for i in $(curl $baseurl | grep iso.torrent | cut -f10 -d'"' ); do

                ##checking weather the torrent file already exists, downloading only if the file doesnt exist 
                if [ -f $i ];
                then
                        #outputing that the file exists
                        echo "file $i already exists"
                else
                        ##downloading the torrentfile
                        wget $baseurl$i
                        ##logging the download
                        echo $i >> $log
                fi

        done


##if a download log exists (will only exist if atleast one .torrent file was downloaded)  
##then I get an email with the content of the log file sent to me , after that that log file is deleted. 
if  [ -f  $log];
        then
                 mail.sh "[email protected]" "ubuntuget output $(date)" " $(cat $log)"
                rm  $log
fi

this is the mail.sh script called in the script above

I had a few scripts that email me their output, and i figured it was easier to make an email script and use that instead of rewriting it every time

Code:

#!/bin/bash
##markp1989 
##small email script

##usage
##mail.sh "recipent" "subject" "body"
## eg mail.sh "[email protected],[email protected]" "Just saying hi" "hey, just emailing you to give you this notification" 


recipent=$1
subject=$2
body=$3

echo -e "$(date) \n\n $body" | mail -s "$subject" $recipent

still pretty new to scripting and programing in general, so any pointers on how to optimize these scripts will be appreciated.

${freak} · Apr 4, 2011

never use 'for' to iterate through the output of a command. for is through iterating over a list of arguments. also, options to echo are not portable. use printf.

and MORE QUOTES. ''rm $log'' is dangerous. ''rm "$log"''

'[' is a command. commands take arguments, separated by whitespace. in this case, it is another way of invoking test(1), except the last argument has to be ']'. it also means that variables inside are subject to wordsplitting, and should therefore be quoted. in bash, though, i prefer to use [[, which is much safer about these things, and has more features, like pattern and regex matching.

Code:

#!/bin/bash
##markp1989
##script to download torrents of the latest ubuntu release. 


##setting the watch folder of your torrent client 
watch=/home/rtorrent/watch

##setting where to log any downloads 
log=/home/rtorrent/downloaded.ubuntuget

##cding to the watch folder  
cd "$watch" || exit

# note: always check the exit status of 'cd', to avoid running stuff in the wrong directory

##scraping part of the releases page to find out the name of the latest version 
newest=$(curl http://releases.ubuntu.com/ | grep ")</a>" | cut -f2 -d'"' | cut -f1 -d'/' | tail -n 1)

# note: bash and co. shouldn't generally be used to parse html, but since i don't know
# what it looks like, i'm not touching this one

##setting the base url  which is http://releases.ubuntu.com/NameOfNewestRelease 
baseurl="http://releases.ubuntu.com/$newest/"

#running a loop for every torrent link on the latest release page
while read -r file; do
  ##checking weather the torrent file already exists, downloading only if the file doesnt exist 
  if [[ -f $file ]]; then
    #outputing that the file exists
    echo "file $file already exists"
  else
    ##downloading the torrentfile
    wget "$baseurl$file"
    ##logging the download
    printf '%s\n' "$file"  >> "$log"
  fi
done < <(curl "$baseurl" | fgrep 'iso.torrent' | cut -f10 -d'"')

##if a download log exists (will only exist if atleast one .torrent file was downloaded)  
##then I get an email with the content of the log file sent to me , after that that log file is deleted. 
if  [[ -f  $log ]]; then
  mail.sh "[email protected]" "ubuntuget output $(date)" " $(<"$log")"
  rm  "$log"
fi

Code:

#!/bin/bash
##markp1989 
##small email script

##usage
##mail.sh "recipent" "subject" "body"
## eg mail.sh "[email protected],[email protected]" "Just saying hi" "hey, just emailing you to give you this notification" 


recipent=$1
subject=$2
body=$3

printf '%s\n\n%s\n' "$(date)" "$body" | mail -s "$subject" "$recipent"

http://mywiki.wooledge.org/BashGuide is the only place out there to learn to write decent bash. to explain the while read loop, read http://mywiki.wooledge.org/BashFAQ/001

http://wiki.bash-hackers.org/commands/builtin/printf is a nice explaination of the printf command.

markp1989 · Apr 4, 2011

${freak} said:
never use 'for' to iterate through the output of a command. for is through iterating over a list of arguments. also, options to echo are not portable. use printf.

and MORE QUOTES. ''rm $log'' is dangerous. ''rm "$log"''

'[' is a command. commands take arguments, separated by whitespace. in this case, it is another way of invoking test(1), except the last argument has to be ']'. it also means that variables inside are subject to wordsplitting, and should therefore be quoted. in bash, though, i prefer to use [[, which is much safer about these things, and has more features, like pattern and regex matching.

Code:

#!/bin/bash ##markp1989 ##script to download torrents of the latest ubuntu release. ##setting the watch folder of your torrent client watch=/home/rtorrent/watch ##setting where to log any downloads log=/home/rtorrent/downloaded.ubuntuget ##cding to the watch folder cd "$watch" || exit # note: always check the exit status of 'cd', to avoid running stuff in the wrong directory ##scraping part of the releases page to find out the name of the latest version newest=$(curl http://releases.ubuntu.com/ | grep ")</a>" | cut -f2 -d'"' | cut -f1 -d'/' | tail -n 1) # note: bash and co. shouldn't generally be used to parse html, but since i don't know # what it looks like, i'm not touching this one ##setting the base url which is http://releases.ubuntu.com/NameOfNewestRelease baseurl="http://releases.ubuntu.com/$newest/" #running a loop for every torrent link on the latest release page while read -r file; do ##checking weather the torrent file already exists, downloading only if the file doesnt exist if [[ -f $file ]]; then #outputing that the file exists echo "file $file already exists" else ##downloading the torrentfile wget "$baseurl$file" ##logging the download printf '%s\n' "$file" >> "$log" fi done < <(curl "$baseurl" | fgrep 'iso.torrent' | cut -f10 -d'"') ##if a download log exists (will only exist if atleast one .torrent file was downloaded) ##then I get an email with the content of the log file sent to me , after that that log file is deleted. if [[ -f $log ]]; then mail.sh "[email protected]" "ubuntuget output $(date)" " $(<"$log")" rm "$log" fi

Code:

#!/bin/bash ##markp1989 ##small email script ##usage ##mail.sh "recipent" "subject" "body" ## eg mail.sh "[email protected],[email protected]" "Just saying hi" "hey, just emailing you to give you this notification" recipent=$1 subject=$2 body=$3 printf '%s\n\n%s\n' "$(date)" "$body" | mail -s "$subject" "$recipent"

http://mywiki.wooledge.org/BashGuide is the only place out there to learn to write decent bash. to explain the while read loop, read http://mywiki.wooledge.org/BashFAQ/001

http://wiki.bash-hackers.org/commands/builtin/printf is a nice explaination of the printf command.

Thanks for replying, thats the sort of info i was after

.

gonna read through the links in a second , Thank you.

${freak} · Apr 4, 2011

by the way, http://mywiki.wooledge.org/BashFAQ/035

this will help explain how to handle options easily. feel free to pm me if you have any questions, or check out #bash on irc.freenode.net

for your case, checking that they exist may suffice, but i would use options. -s for the subject, -b for the body or let it read from stdin, -r for the recipient. it would probably be something like:

Code:

die() { printf "%s\n" "$@" >&2; exit 1; }

while [[ $1 = -* ]]; do
  case $1 in
    -s)
      if [[ $2 != -* ]]; then
        subject=$2
        shift 2
      else
        die "-s requires subject argument"
      fi
      ;;
    -r)
      if [[ $2 != -* ]]; then
        recipient=$2
        shift 2
      else
        die "-r requires recipient argument"
      fi
      ;;
    -b)
      if [[ $2 = - ]]; then
        if [[ -t 0 ]]; then
          die "nothing on stdin"
        fi
        body=$(cat)
        shift
      elif [[ $2 != -* ]]; then
        body=$2
        shift 2
      else
        die "-b requires body" "use '-b -' to read from stdin."
      fi
      ;;
esac
if [[ -z $subject || -z $body || -z $recipient ]]; then
  die "requires subject, body, and recipient"
fi

you would probably also want a usage() function, and a -h or --help. but that would likely be the basic idea

of course, you could use a regex to validate the email address, etc, it all depends on how complicated you want to make it.

suggested improvements to these scripts

markp1989

Member

${freak}

Member

markp1989

Member

${freak}

Member

Similar threads