Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[Solved] Unix Pipe Losing Data?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
jasealpers
n00b
n00b


Joined: 02 Jan 2006
Posts: 29

PostPosted: Thu Jun 26, 2014 7:29 pm    Post subject: [Solved] Unix Pipe Losing Data? Reply with quote

Hello,

I'm working on a script to post-process mythtv recordings and load them into plex. The purpose of the script doesn't really matter, but the transcoding takes a long time. The problem I'm running into is that when I try to iterate through a bunch of input files, it seems like the pipe loses some data. Here is how I'm invoking the script...

Code:

ls /mythtv/*.mpg | while read file; do
  echo $file
  myth2plex --debug -i $file
done


Here is the output...

Code:

/mythtv/1003_20070430190000.mpg
Parms = --debug -i /mythtv/1003_20070430190000.mpg
File = /mythtv/1003_20070430190000.mpg
DEBUG 20140626130119: Input file = /mythtv/1003_20070430190000.mpg
DEBUG 20140626130121: Plex name = /plex/How I Met Your Mother/Season 2/How I Met Your Mother S02E20 (Showdown)
DEBUG 20140626130121: Commercial skipping for 1003_20070430190000.mpg; chanid = 1003 starttime = 20070501000000
DEBUG 20140626130917: Original duration = 1794; new duration = 1161
DEBUG 20140626130917: Encoding video for /mythtv/tmp/7225.nocomm.mpg
DEBUG 20140626132653: Copying audio for /mythtv/tmp/7225.nocomm.mpg
DEBUG 20140626132656: Muxing output /plex/How I Met Your Mother/Season 2/How I Met Your Mother S02E20 (Showdown).mkv
7190000.mpg
Parms = --debug -i 7190000.mpg
File = 7190000.mpg
DEBUG 20140626132704: Input file = 7190000.mpg
ERROR: 7190000.mpg not in mythtv database
/mythtv/1003_20070514190000.mpg
Parms = --debug -i /mythtv/1003_20070514190000.mpg
File = /mythtv/1003_20070514190000.mpg
DEBUG 20140626132704: Input file = /mythtv/1003_20070514190000.mpg
DEBUG 20140626132706: Plex name = /plex/How I Met Your Mother/Season 2/How I Met Your Mother S02E22 (Something Blue)
DEBUG 20140626132706: Commercial skipping for 1003_20070514190000.mpg; chanid = 1003 starttime = 20070515000000
DEBUG 20140626133510: Original duration = 1796; new duration = 1796
DEBUG 20140626133510: Encoding video for /mythtv/tmp/8720.nocomm.mpg
DEBUG 20140626135821: Copying audio for /mythtv/tmp/8720.nocomm.mpg
DEBUG 20140626135832: Muxing output /plex/How I Met Your Mother/Season 2/How I Met Your Mother S02E22 (Something Blue).mkv


myth2plex is the script I've written. I'm not sure if it's something I'm doing in my script or if the long duration (over an hour per myth2plex execution) causes the pipe to lose position somehow?

If I just execute

Code:

ls /mythtv/*.mpg | while read file; do
  echo $file
done


I get something like...

Code:

/mythtv/1003_20070430190000.mpg
/mythtv/1003_20070507190000.mpg
/mythtv/1003_20070514190000.mpg


You can see that somehow the second file gets modified when the myth2plex script is called. Sometimes it's just one file. Sometimes I lose multiple files and then see the last one in the group show up mutilated like above.

Any ideas why this is happening? Anything I could look for in my script that might be the culprit?

Thanks!
Jase


Last edited by jasealpers on Fri Jun 27, 2014 8:07 pm; edited 1 time in total
Back to top
View user's profile Send private message
russK
l33t
l33t


Joined: 27 Jun 2006
Posts: 665

PostPosted: Thu Jun 26, 2014 8:17 pm    Post subject: Reply with quote

Can your script handle spaces in file names? I expect you only want to process mythtv recordings, is there anything else that the 'ls' command might pick up?

Maybe you could try a find command:

Code:
find /mythtv/ -maxdepth 1 -type f -iname "*.mpg" -exec myth2plex --debug -i "{}" \;
Back to top
View user's profile Send private message
lagalopex
Guru
Guru


Joined: 16 Oct 2004
Posts: 565

PostPosted: Thu Jun 26, 2014 11:06 pm    Post subject: Re: Unix Pipe Losing Data? Reply with quote

jasealpers wrote:
Code:

ls /mythtv/*.mpg | while read file; do
  echo $file
  myth2plex --debug -i $file
done


Perhaps myth2plex reads from stdin? You could try to redirect something. :)

(Or use the suggestion from russK.)
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Thu Jun 26, 2014 11:48 pm    Post subject: Reply with quote

I'll put my bet that $file is also used inside myth2plex...
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 22657

PostPosted: Fri Jun 27, 2014 1:55 am    Post subject: Reply with quote

Never use ls to find files programmatically. In shell, quote your variables unless you know why you need it unquoted. When processing filenames, use null separators. The example shown by russK is a good one if you do not need to use the filename in multiple ways.
Back to top
View user's profile Send private message
vaxbrat
l33t
l33t


Joined: 05 Oct 2005
Posts: 731
Location: DC Burbs

PostPosted: Fri Jun 27, 2014 2:16 am    Post subject: Never say never :D Reply with quote

There are times when I substitute os.walk in python with a subprocess call to "ls -1" since I don't want to descend into subdirs. The "-1" switch tells ls to put one file per line so then I just do a str.split('\n') on the output and I don't have to worry about the quoting hassles on anything other than what I might be passing as an arg to ls.
Back to top
View user's profile Send private message
jasealpers
n00b
n00b


Joined: 02 Jan 2006
Posts: 29

PostPosted: Fri Jun 27, 2014 3:10 pm    Post subject: Reply with quote

Thanks very much for all the responses.

$file is used in the script, but I typeset the variables in the script just in case and also tried modifying the invocation to use $f instead...

Code:

ls /mythtv/*.mpg | while read f; do
  echo $f
  myth2plex --debug -i $f
done


but this didn't make any difference. It seems more likely the script is somehow reading from stdin, but I can't find where that might happen.

Here is the script:

Code:

#!/bin/sh

trap 'rm -f $tempdir/$$.*' 0 1 2 3 13 15

##################################################################################
# Settings
##################################################################################
typeset plexdir="/plex"

# Myth database settings
typeset mysql_host=xxx
typeset mysql_db=mythconverg
typeset mysql_user=user
typeset mysql_pass=pass
typeset mysql="mysql -h $mysql_host -u $mysql_user -p$mysql_pass -D $mysql_db --column-names=false -se"

# TVDB Settings
typeset tvdburl="www.thetvdb.com/api"
typeset apikey="79A63A422032F6FE"

# Encoding options
typeset encode_audio=0
typeset x264encopts="crf=23:threads=$(nproc):subq=5:8x8dct:frameref=2:bframes=3:b_pyramid=normal:weight_b"
typeset video_filter="yadif,harddup"
typeset ogg_quality=3

# Temporary files
typeset tempdir=/mythtv/tmp
typeset xml=$tempdir/$$.xml
typeset avi_file=$tempdir/$$.avi
typeset ac3_file=$tempdir/$$.ac3
typeset wav_file=$tempdir/$$.wav
typeset ogg_file=$tempdir/$$.ogg
typeset nocomm_file=$tempdir/$$.nocomm.mpg

##################################################################################
# Functions start here
##################################################################################
function DEBUG() {
  if [[ $debug -eq 1 ]]; then
    echo "DEBUG $(date +"%Y%m%d%H%M%S"): $1"
  fi
}

function showHelp() {
  echo "myth2plex"
  echo
  echo "Post-processes a mythtv recording for plex.  Includes the following steps:"
  echo "  * Determine plex naming based on mythtv database"
  echo "  * Mark and remove commercials"
  echo "  * Transcode to h.264 for space reduction"
  echo
  echo "Options:"
  echo " -i OR --input-file            Provide the input file for processing (required)"
  echo " -h OR --help                  Display this help printout"
  echo " -n OR --no-processing         Determine the new filename only, no processing"
  echo " --overwrite                   Overwrite existing file"
  echo " --no-remove-commercials       Don't remove commercials"
  echo " --no-encoding                 Don't reencode to h.264"
}

function getDuration {
  ffprobe -show_format $1 2>/dev/null | grep duration |
    awk -v FS== '{print $2}' | awk -v FS=. '{print $1}'
}

function matchShow() {
  input_file=$1
  series=$($mysql "select title from recorded where basename='$input_file';")
  if [[ $series = "" ]]; then
    echo "ERROR: $input_file not in mythtv database" >&2
    return 1
  fi
  tvdb_series=$(echo $series | sed s/" "/"%20"/g | sed s/"&"/"%26"/g)
  episode=$($mysql "select subtitle from recorded where basename='$input_file';")
  origairdate=$($mysql "select originalairdate from recorded where basename='$input_file';")

  curl -L -s -m10 "$tvdburl/GetSeries.php?seriesname=$tvdb_series" > $xml

  if [[ $(xml sel -t -v "count(//seriesid)" -n $xml) -eq 1 ]]; then
    seriesid=$(xml sel -t -v "//seriesid" -n $xml)
  else
    seriesid=$(xml sel -t -m "//SeriesName[translate(.,
                                                     'ABCDEFGHIJKLMNOPQRSTUVWXYZ',
                                                     'abcdefghijklmnopqrstuvwxyz') =
                                           translate('$series',
                                                     'ABCDEFGHIJKLMNOPQRSTUVWXYZ',
                                                     'abcdefghijklmnopqrstuvwxyz')]" \
                       -v ../seriesid -n $xml)
    if [[ $? -ne 0 ]]; then
      echo "ERROR: Series $series not found" >&2
      return 1
    fi
  fi

  curl -L -s -m10 "$tvdburl/GetEpisodeByAirDate.php?apikey=$apikey&seriesid=$seriesid&airdate=$origairdate" > $xml

  tvdb_episode=$(xml sel -t -m '//EpisodeName' -v . -n $xml)
  if [[ $? -ne 0 ]]; then
    echo "ERROR: Airdate $origairdate not found for series $series ($seriesid) ($input_file)" >&2
    return 1
  fi

  Sxx=$(xml sel -t -m '//SeasonNumber' -v . -n $xml)
  season=$Sxx
  Exx=$(xml sel -t -m '//EpisodeNumber' -v . -n $xml)

  test $Sxx -lt 10 && Sxx="S0$Sxx" || Sxx="S$Sxx"
  test $Exx -lt 10 && Exx="E0$Exx" || Exx="E$Exx"

  if [[ $episode == "" ]]; then
    episode=$tvdb_episode
  fi
  episode=$(echo $episode | sed "s/\//,/g")     # Char / in episode names causes new directory
  dir="$plexdir/$series/Season $season"
  mkdir -p "$dir"
  output_name="$dir/$series $Sxx$Exx ($episode)"

  echo $output_name
}

function removeCommercials() {
  input_file=$1
  chanid=$($mysql "select chanid from recorded where basename='$input_file';")
  starttime=$($mysql "select starttime from recorded where basename='$input_file';" |
              tr -d " :-")
  DEBUG "Commercial skipping for $input_file; chanid = $chanid starttime = $starttime"

  mythcommflag --method 7 --noprogress --chanid $chanid --starttime $starttime >> $log_file 2>&1
  mythutil --gencutlist --chanid $chanid --starttime $starttime >> $log_file 2>&1
  mythtranscode --mpeg2 --honorcutlist -i /mythtv/$input_file -o $nocomm_file >> $log_file 2>&1

  # Check the resulting duration to make sure we didn't cut almost everything
  orig_duration=$(getDuration /mythtv/$input_file)
  new_duration=$(getDuration $nocomm_file)
  DEBUG "Original duration = $orig_duration; new duration = $new_duration"
  let "limit = orig_duration/2"

  if [[ $new_duration -lt $limit ]]; then    # Less than half the length
    # Try again with less aggressive commercial cutting
    DEBUG "Retrying with less aggressive commercial skipping"
    mythcommflag --method 3 --noprogress --chanid $chanid --starttime $starttime >> $log_file 2>&1
    mythutil --gencutlist --chanid $chanid --starttime $starttime >> $log_file 2>&1
    mythtranscode --mpeg2 --honorcutlist -i /mythtv/$input_file -o $nocomm_file >> $log_file 2>&1
  fi

  currfile=$nocomm_file
}

function encodeFile() {
  input_file=$1
  output_file=$2

  DEBUG "Encoding video for $input_file"
  mencoder -oac pcm -vf $video_filter \
           -ovc x264 -x264encopts $x264encopts \
           -quiet -o $avi_file $input_file >> $log_file 2>&1

  if [[ $encode_audio -eq 0 ]]; then
    DEBUG "Copying audio for $input_file"
    ffmpeg -i $input_file -c:a copy -vn $ac3_file >> $log_file 2>&1

    DEBUG "Muxing output $output_file"
    mkvmerge -o "$output_file" -A $avi_file $ac3_file >> $log_file 2>&1
  else
    DEBUG "Encoding audio for $input_file"
    mplayer $input_file -ao pcm:file=$wav_file -channels 6 -vc dummy -vo null >> $log_file 2>&1
    oggenc -q $ogg_quality $wav_file $ogg_file >> $log_file 2>&1

    DEBUG "Muxing output $output_file"
    mkvmerge -o "$output_file" -A $avi_file $ogg_file >> $log_file 2>&1
  fi
}

##################################################################################
# Mainline code starts here
##################################################################################
typeset script_name=${0##*/}
typeset GNUOPT=$(getopt -o hni: --long help,debug,no-processing,overwrite,no-remove-commercials,no-encoding,input-file: \
             -n "$script_name" -- "$@")

if [[ $? -ne 0 ]]; then
  echo
  showHelp
  exit 1
fi

eval set -- "$GNUOPT"

typeset debug=0
typeset noprocess=0
typeset overwrite=0
typeset removeComm=1
typeset encoding=1
typeset file=
typeset newname=
typeset log_file=
typeset currfile=
while true; do
  case "$1" in
    --debug ) debug=1; shift ;;
    -n | --no-processing ) noprocess=1; shift ;;
    --overwrite ) overwrite=1; shift ;;
    --no-remove-commercials ) removeComm=0; shift ;;
    --no-encoding ) encoding=0; shift ;;
    -i | --input-file ) file="$2"; shift 2 ;;
    -h | --help ) showHelp; exit 0; shift ;;
    -- ) shift; break ;;
    * ) break ;;
  esac
done

if [[ $file == "" ]]; then
  echo "$script_name: --input-file required"
  echo
  showHelp
  exit 1
fi

DEBUG "Input file = $file"

file=${file##*/}
log_file=$tempdir/$file.log

# Get new file name based on tvdb lookup
newname=$(matchShow $file)
if [[ $? -ne 0 ]]; then
  exit 1
fi

# If we're just getting the name, then exit
if [[ $noprocess -eq 1 ]]; then
  echo $newname
  exit 0
fi

# Don't overwrite existing file unless forced
if [[ $overwrite -eq 0 ]]; then
  if [[ -e "$newname.mkv" || -e "$newname.mpg" ]]; then
    echo "Not overwriting $newname"
    exit 0
  fi
fi

DEBUG "Plex name = $newname"

# Remove commercials if requested
currfile=$file
if [[ $removeComm -eq 1 ]]; then
  removeCommercials $file
fi

# Do encoding if requested
if [[ $encoding -eq 1 ]]; then
  encodeFile $currfile "$newname.mkv"
else
  cp $currfile "$newname.mpg"
fi


I used the circumvention from russK and that has successfully avoided the problem.

I'm still not sure why the pipe method doesn't work, but maybe that doesn't matter so much anymore. :-)

Thanks again!
Jase
Back to top
View user's profile Send private message
pa4wdh
l33t
l33t


Joined: 16 Dec 2005
Posts: 882

PostPosted: Fri Jun 27, 2014 4:50 pm    Post subject: Reply with quote

I know from using ffmpeg in a script that it reads from stdin even when you don't expect it, and i see you also use it in your script.
The way to stop it is to redirect stdin to /dev/null.

So your line containing ffmpeg is now
Code:

    ffmpeg -i $input_file -c:a copy -vn $ac3_file >> $log_file 2>&1

and should be:
Code:

    ffmpeg -i $input_file -c:a copy -vn $ac3_file < /dev/null >> $log_file 2>&1


If you don't care which part of your script does this, you can of course do the same with your script and change:
Code:

 myth2plex --debug -i $f

to
Code:

 myth2plex --debug -i $f < /dev/null

_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world

My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Fri Jun 27, 2014 7:34 pm    Post subject: Reply with quote

jasealpers wrote:
Code:
#!/bin/sh

jasealpers ... if you are going to set the script to the posix shell then please don't use bashisms such as trap (with signals), typeset, function, double square brace test condtions, let, and '$( ... )" substitutions (though I'm not sure any current posix shells with have an issue with the latter).

jasealpers wrote:
Code:
function getDuration {
  ffprobe -show_format $1 2>/dev/null | grep duration |
    awk -v FS== '{print $2}' | awk -v FS=. '{print $1}'
}

There isn't any need to have grep and then call awk twice, awk provides standard printf '%d' to get to the decimal point, and can match pattern ...

Code:
getDuration {
  ffprobe -show_format "$1" 2>/dev/null | awk -F'=' '/duration/{printf ("%d\n",$2)}'
}

jasealpers wrote:
Code:
typeset script_name=${0##*/}

This is also a bashism ... the posix equivilant would be to use basename.

Code:
script_name="`basename -- $0`"

jasealpers wrote:
Code:
file=${file##*/}

... similarly ...

Code:
file="`basename -- $1`"

best ... khay
Back to top
View user's profile Send private message
jasealpers
n00b
n00b


Joined: 02 Jan 2006
Posts: 29

PostPosted: Fri Jun 27, 2014 8:06 pm    Post subject: Reply with quote

pa4wdh...Thank you very much! Exactly as you said, ffmpeg was eating stdin. Your suggested change solved the problem. I would have never thought ffmpeg would be the culprit. Much appreciated!
Back to top
View user's profile Send private message
russK
l33t
l33t


Joined: 27 Jun 2006
Posts: 665

PostPosted: Fri Jun 27, 2014 8:19 pm    Post subject: Reply with quote

You must have had some files with spaces.

You might have been able to work around it with this:

Code:
ls *.mpg | while read f; do
   echo $f
  mythh2plex --debug -i "$f"
done


Note the quotes around $f. With no quotes around "$f", the filename will be split into pieces with the spaces as delimiter. So in the while loop here, $1 will see multiple pieces of the file name instead of the whole thing.

Code:

while true; do
  case "$1" in
    --debug ) debug=1; shift ;;
    -n | --no-processing ) noprocess=1; shift ;;
    --overwrite ) overwrite=1; shift ;;
    --no-remove-commercials ) removeComm=0; shift ;;
    --no-encoding ) encoding=0; shift ;;
    -i | --input-file ) file="$2"; shift 2 ;;
    -h | --help ) showHelp; exit 0; shift ;;
    -- ) shift; break ;;
    * ) break ;;
  esac
done


By the way, the $file inside myth2plex is different from the $file or $f outside the script.


HTH
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 22657

PostPosted: Sat Jun 28, 2014 12:19 am    Post subject: Re: Never say never :D Reply with quote

vaxbrat wrote:
There are times when I substitute os.walk in python with a subprocess call to "ls -1" since I don't want to descend into subdirs. The "-1" switch tells ls to put one file per line so then I just do a str.split('\n') on the output and I don't have to worry about the quoting hassles on anything other than what I might be passing as an arg to ls.
Why not use os.listdir? That saves you a subprocess call and handles strange filenames correctly. Remember, the only characters you can assume do not appear in a filename are null and slash.

The correct way to iterate over files in a shell script is either to rely on a shell glob or to use find rules -print0 | while read -d '' f. Shell globs may fail when the number of matches is very large.
Back to top
View user's profile Send private message
vaxbrat
l33t
l33t


Joined: 05 Oct 2005
Posts: 731
Location: DC Burbs

PostPosted: Sat Jun 28, 2014 3:00 am    Post subject: must have gotten snuck in there on me Reply with quote

I never noticed when that snuck into the os module. Maybe it was when os.walk got overhauled some time back?
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 22657

PostPosted: Sat Jun 28, 2014 2:57 pm    Post subject: Reply with quote

I do not know when it was added. The "What's New" documents mention changes in the behavior of os.listdir as early as 2.3 when Python/Windows NT gained Unicode support for os.listdir.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Sun Jun 29, 2014 2:28 am    Post subject: Reply with quote

khayyam wrote:
if you are going to set the script to the posix shell then please don't use bashisms such as trap (with signals), typeset, function, double square brace test condtions, let, and '$( ... )" substitutions (though I'm not sure any current posix shells with have an issue with the latter).

trap works in sh, though you should use eg HUP instead of SIGHUP (latter is allowed, but may equally be ignored as an extension.)
Every sh is going to treat '$( ... )' in single-quotes as raw, but $(..) expansions have been in POSIX since at least 1996.
Quote:
jasealpers wrote:
Code:
typeset script_name=${0##*/}

This is also a bashism ... the posix equivilant would be to use basename.

No, it's not (apart from typeset, without going into *ksh.) It's a POSIX sh parameter expansion.

basename is slower, and not needed for this. The `..` format has been deprecated for ages, as mentioned. It really sucks. It's only autoconf that keeps it zombiefied, afaic (along with its awful use of [ ] quoting.)

@jasealpers: you should lose the typeset and other things khayyam mentions, and ofc switch to awk, but most importantly you really need to start quoting parameter expansions (as does pa4wdh.) The script is not reusable as it is.

As for the parms, you might find this informative; yours is similar, though you can just do the loop without forking to GNU getopt --long. You just call usage (or showHelp) whenever you find an unrecognised option. It seems odd to use the external, only to wipe the settings and repeat the logic.

In fact it shows us the only two places you don't need to quote a parameter expansion; case $foo in and: foo=$bar (but only when a standard assignment, not in say: export) are defined not to field-split (by POSIX, so applies to sh, as well as bash et al.) You do ofc need quotes if you have spaces or shell meta chars/operators in the text.

Also you can do multiple assignments in one statement; POSIX specifies that they occur in the current execution environment, when they are not followed by a cmd (where they would be in effect only for the environment where the command is run, or while the function is running.) So:
Code:
debug=0 noprocess=0 overwrite=0 removeComm=1 encoding=1 log_file=
would normally go at the top of the script (since it makes it easier to see what is important, and can be overridden by a .conf file if you move to having one.)
Code:
file= newname= currfile=
would typically go just before the loop to check flags, where you have them, since they're specific to the invocation.

If you want default settings the user can override from the environment you'd use:
Code:
: "${debug:=0} ${noprocess:=0} ${overwrite:=0} ${removeComm:=1} ${encoding:=1}"
Think carefully before you do, especially with something that is meant to be numeric, and don't do that kind of thing for scripts you run as root.

Respond to environment vars for other programs you call (as in be aware of them, and check their validity), sure, but as a rule, if your script is used for system-maintenance keep its settings fixed at the top, and dot (the . command, equivalent to bash source) a known config file for something larger. An admin can edit the script if needed, and OS-specific defaults can be done as part of make install. Providing the script is robust, or there's no point worrying about any of the rest.

[[ $series = "" ]] is clearer as [[ -z $series ]] or [ -z "$series" ] in sh.

If you want more help, use IRC: chat.freenode.net and /join #bash -- they'll help you with sh too, if you make it clear you're writing sh upfront.

Here's some links in case you've not seen them before:
http://wiki.bash-hackers.org/doku.php?id=scripting:basics | http://mywiki.wooledge.org/BashGuide
http://www.grymoire.com/Unix/index.html | http://mywiki.wooledge.org/BashFAQ
http://www.shelldorado.com/ | http://mywiki.wooledge.org/BashPitfalls

man bash is ofc the goto command, but you can also use help at the cmd-line, eg help test

HTH,
steveL.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Sun Jun 29, 2014 1:41 pm    Post subject: Reply with quote

steveL wrote:
trap works in sh, though you should use eg HUP instead of SIGHUP (latter is allowed, but may equally be ignored as an extension.)

steve ... yes, but I did add the priviso "with signals" as the base specification states "the output of historical trap commands is not portable (because numeric signal values are not portable)".

steveL wrote:
Every sh is going to treat '$( ... )' in single-quotes as raw, but $(..) expansions have been in POSIX since at least 1996.

The single quote is a typo, it closes with a double quote. Anyhow, yes, I wasn't quite sure whether this was posix or not, which is why I added "though I'm not sure any current posix shells with have an issue with the latter".

steveL wrote:
Quote:
jasealpers wrote:
Code:
typeset script_name=${0##*/}

This is also a bashism ... the posix equivilant would be to use basename.

No, it's not (apart from typeset, without going into *ksh.) It's a POSIX sh parameter expansion.

Indeed ... I stand corrected. I *always* err with parameter expansion having been caught by ${!param}

steveL wrote:
@jasealpers: you should lose the typeset and other things khayyam mentions, and ofc switch to awk [...]

That was my only reason for posting the above ... correcting the awk oneliner ... but as jasealpers hasn't commented I assume that was lost in the point about using /bin/sh when /bin/bash is intended.

best ... khay
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum