Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
portable sed in place editing
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Dominique_71
Veteran
Veteran


Joined: 17 Aug 2005
Posts: 1895
Location: Switzerland (Romandie)

PostPosted: Thu Nov 14, 2024 1:00 pm    Post subject: portable sed in place editing Reply with quote

Something that was not obvious to me to find. It is 3 possibilities to make an in place file sed editing:

    1) On linux, we can use:
    Code:
    sed -i 'sed command' file

    If file don't exist, no empty file will be created. But that will not work on some systems, as example freebsd.

    2) On both linux and freebsd, we can add a .suffix to -i:
    Code:
    sed -i.bak 'sed command' file

    If file don't exist, no empty file will be created.

    3) It is not in place editing but the result is the same, and it is the safest way to do it - on both linux and freebsd, we can use redirection and a temporary file:
    Code:
    sed 'sed command' file > file.tmp
    mv -f file.tmp file

    or
    Code:
    sed 'sed command 1' file > file.tmp
    sed 'sed command 2' file.tmp > file

    If file don't exist, that will create an empty file, which can cause nasty bugs. That imply it is best and necessary to test if file exist and run these 2 commands only if it exist.

_________________
"Confirm You are a robot." - the singularity
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 22654

PostPosted: Thu Nov 14, 2024 1:27 pm    Post subject: Reply with quote

If you really don't want to rely on GNU sed, you could rely on redirection ordering instead.
Code:
sed -e p < file > file.tmp && mv file.tmp file
I prefer expecting -i to work, but at least the form I show has the advantage that if the input does not exist, you get a shell error redirecting the input before it ever creates the output, so you don't get the junk temporary file. It also solves a minor TOCTOU issue, by not trying to access the source filename twice (once in the test, again inside sed).
Back to top
View user's profile Send private message
Dominique_71
Veteran
Veteran


Joined: 17 Aug 2005
Posts: 1895
Location: Switzerland (Romandie)

PostPosted: Thu Nov 14, 2024 5:36 pm    Post subject: Reply with quote

Thanks for the tips Hu. As fvwm-crystal's main contributor, I was asked if it can support freebsd and is looking at it. Almost all of the needed fixes was a few path changes with the Makefile, and some of the sed calls.
_________________
"Confirm You are a robot." - the singularity
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 22654

PostPosted: Thu Nov 14, 2024 6:49 pm    Post subject: Reply with quote

I think most of the BSDs can have GNU sed, though they tend to install it as gsed, reserving the name sed for the BSD sed. Perhaps you would be better off just using gsed -i instead of adding logic to work around BSD sed not supporting -i? At least on Gentoo, gsed exists as an alias for the GNU sed installed as sed, so if you use gsed, you can do that without regard to BSD vs Gentoo. I cannot comment on whether other Linux distributions provide gsed as an alias.
Back to top
View user's profile Send private message
Dominique_71
Veteran
Veteran


Joined: 17 Aug 2005
Posts: 1895
Location: Switzerland (Romandie)

PostPosted: Fri Nov 15, 2024 9:36 am    Post subject: Reply with quote

Hu wrote:
I think most of the BSDs can have GNU sed, though they tend to install it as gsed, reserving the name sed for the BSD sed. Perhaps you would be better off just using gsed -i instead of adding logic to work around BSD sed not supporting -i? At least on Gentoo, gsed exists as an alias for the GNU sed installed as sed, so if you use gsed, you can do that without regard to BSD vs Gentoo. I cannot comment on whether other Linux distributions provide gsed as an alias.


Sure. I don't know either for the other distributions. Which imply the safest way is to not use gsed explicitly and change the few calls that are not already compatible with the freebsd version. That will also insure the POSIX.2 compatibility according to sed's man page on freebsd. Which in turn should imply no maintenance on the long run.
_________________
"Confirm You are a robot." - the singularity
Back to top
View user's profile Send private message
eschwartz
Developer
Developer


Joined: 29 Oct 2023
Posts: 227

PostPosted: Fri Nov 15, 2024 4:02 pm    Post subject: Reply with quote

Hu wrote:
I think most of the BSDs can have GNU sed, though they tend to install it as gsed, reserving the name sed for the BSD sed. Perhaps you would be better off just using gsed -i instead of adding logic to work around BSD sed not supporting -i? At least on Gentoo, gsed exists as an alias for the GNU sed installed as sed, so if you use gsed, you can do that without regard to BSD vs Gentoo. I cannot comment on whether other Linux distributions provide gsed as an alias.


Various distros install g* symlinks for some or all GNU utilities, various other distros don't. There's no particular guarantee of anything.

What you could do is check if "command -v gsed" and then use that where possible, but fall back to "sed" if it isn't possible and simply hope that gsed is installed on all systems where the default sed isn't the GNU one.

Note: BSD sed supports -i just fine. That is the entire problem. Some sed implementations support "sed -i pattern file" to write in-place, some support "sed -i .backupname pattern file", and it is impossible to support both approaches, so as a result it's effectively impossible for POSIX to standardize the -i flag.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum