Friday, October 26, 2007

Unix Command to Find And Replace Text in a File

The following is a very simple, one-line command for finding and replacing all instances of a text pattern in a file (or files). There are numerous sources for this short script. It may seem completely trivial, but it comes in handy.
  • perl -pi -e 's/<find>/<replace>/' FILE_1 FILE_2 ... FILE_n

This will only replace the first occurrence of the pattern per line. If you want to replace all occurrences in every line, then add the 'g' option, like this.
  • perl -pi -e 's/<find>/<replace>/g' FILE_1 FILE_2 ... FILE_n

If you want the find to be case-insensitive, then use the 'i' option, like this.
  • perl -pi -e 's/<find>/<replace>/i' FILE_1 FILE_2 ... FILE_n

If you find that you need to replace file path names in text, then instead of back-slashing all the path separators in your pattern, you might want to choose a different s/// separator. I have found that '~' works pretty well (but if your paths contain '~'s, then you should experiment).
  • perl -pi -e 's~/path/to/my/file~/new/path/to/file~' FILE_1 FILE_2 ... FILE_n

If you want to replace some text with the contents of an environment variable, then you can access the environment with the hash ENV.

  • perl -pi -e 's~some_text~$ENV{environment_variable}~' FILE_1 FILE_2 ... FILE_n


joseph said...

It came up as my first google and I used it to prepare my deep sequencing files for downstream analysis.
Thank you very much for your help.
Joseph Dhahbi
CHORI Oakland CA

Tim Myer said...

I appreciate the feedback and am happy to be of help.

Sampath Peechu said...

Nice, it helped me avoid monotony and save time.

bala said...

This was so useful. Thanks

Mangal said...

I need to find following words "ABC Category Code Description" and "ABC Category Code" and replace it with "XYZ Category Code Description" and "XYZ Category Code" respectively. How your script to be modified? My dilemma is that one string to be replaced is subset of the another. Any help will be greatly appreciated.

Tim Myer said...

Hi Mangal,

I am not sure why I was not alerted to this comment, but hopefully this reply will be of use to someone else.
My inclination would be to do two find and replace operations: the first using the super-string and the second using the sub-string (which will no longer be a sub-string after the first replacement operation).


pavan said...

nice command . I used it first time

Sri said...

Hey, its a very nice blog.
I've a scenario but am not sure how to solve this.
I've two files;
A.txt (contents aaa)
and B.txt (contents bbb)
I need to find the string 'aaa' from A.txt and after finding, I've to add string 'bbb' from B.txt
Finally, contents of A.txt should be 'aaabbb'
Any help would be much appreciated

Tim Myer said...

Hi Sri,
If you are just looking to concatenate the contents of one file onto another perhaps you could just use 'cat'.

For example, would 'cat A.txt B.txt > A.txt' be sufficient?

If you do not want line separators, then you could also use 'cat A.txt B.txt | tr -d '\n' > A.txt'

I hope that helps,

Sri said...

Thanks, Tim.
Am not exactly looking for concatenating.

Simply, find a text in one file and add the text from another file.
Contents: 1 2 3 4 5 6
Contents: 7 8 9 10 11 12

Search A for '3' and add the contents from B.
Final output: 1 2 3 7 8 9 10 11 12 4 5 6

Tim Myer said...

Hi Sri,

Thanks for clarifying. Perhaps this will help:
B_CONTENTS=`cat B.txt` perl -pi -e 's~3~3 $ENV{B_CONTENTS}~' A.txt

Sri said...

Yeah, Tim.

I hope, this solves my problem.

Just a small clarification; This will add the text from A to B or from B to A..??
Question might be silly, forgive my ignorance.

Tim Myer said...

Hi Sri,
Why not try it out with the test case you gave me in order to answer your own question ;)

Sri said...

Sure Tim, I'll give a try.

Thanks for your time and efforts.
I really appreciate.

This is a very good blog.
-- Keep Going..!!