Batch edit files

Ok quick ques (2 today from me oO)

once again am going back to site have/am designing.

I want to change an element of each page - the code for each os the same

basically it will be a straight find/replace on each page

the code to be replaced is the same on each page as is the new code am writing in its place.

Now i could go through each page 1 at a time and do a ctrl+h but that would take a while. Is there anyway to automate it?

Like a program i could load all the files that need changed into specify the text to find what to replace it with and let it do the hard work?

v_Ln

oh one thing i forgot to add - have tried some search/replace programs which handle multiple files but my problem is that the text

Code:

---

        <div id="container">
                <div id="menu">
                        <a href="index.php">Home</a>
                        <a href="lab.php">The Lab </a>
                        <a href="archive.php">Archives</a>
                        <a href="digiart.php">Digiart</a>
                        <a href="b3ta.php">b3ta</a>
                        <a href="photos.php">Photos</a>
                <a href="contact.php">Contact</a></div>

---

stretches over multiple lines and most only aloow me to paste in the code up until the first 'return'

v_Ln

Hi valhallen

In the past, before php, dreamweaver and such, I did create
a small c program, which more or less does what you want to do
now. Find it attached (compiles as well under linux as windows).
But - before you use it: create a backup (the program itself
creates a backup before changing the files, but one never knows...)!

I am not aware of some tools or scripts existing which are doing
this. To write a script takes quite some effort, I think (hence, a
c (or perl) program).

Nowadays, if php is enabled, I would do the following, for future
references:

1) store that whole navigation part in a file "navigation.php"
2) change the text in the files to

Code:

---

<?php
        include ('navigation.php');
?>

---

3) In case of any upcoming change, you have to change one file only.


Cheers

What I would suggest is that if you have code that repeats (such as a menu that appears on many pages), you could always separate it out and do an include statement in your html files (forgot the syntax, it's been years!). Not only does this make it more modular and easier for you to change things in the future, it is nowadays considered proper html coding.

You can check out many sites to see how this can be done:

www.w3schools.com
www.devguru.com (my personal favourite)

... and pretty much any html tutorial site that explains some CSS will help you.

hope this helps.
-ik

I think you mean the php include(); function..

Btw, V_Ln if you need it bad, see me in IRC tomorrow, I'll whip you up a bash+sed script to do the job..

thanks for the replies guys

as for the php includes - thats what am trying to replace it with - lol

i noramlly do code all static elements which are seen on every page (like navigation) as a sperate file which is pulled in at runtime....but i didn't this time as I didn't foresee the site being all that large - guess who was wrong! So i thought I'd better do it now before it got any bigger - lol

will take a look at your prog hopefully tomorrow sec - thanks

v_Ln

By popular request..

Code:

---

#!/bin/bash

# Bash multi line remove and replace with new line(s)..
# Will seek first instance of end starting with start.
# Creates filename.bak backup file.
#
# (c) 2005 Anne Jan Brouwer (the_JinX[at]etv.cx)
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# http://www.gnu.org/copyleft/gpl.html

# Define Start element for replace
start='<div id="container">'

# Define End element for replace
end='</div>'

# Replacement string (can be multiline)
replace="<?php
  include( newstuff.php );
?>"

if [[ $* == ""  ]]
then
    echo "Usage: $0 file(s)"
    exit 1
fi

# setup integers
declare -i startline
declare -i endline
declare -i tail
declare -i eof

for phpfile in $*
do
  if [[ `grep "$start" "$phpfile"`  ]]
  then
      mv "$phpfile" "$phpfile.bak"
      eof=`wc -l "$phpfile.bak" | sed -e "s/$phpfile.bak//"`
      startline=`/usr/bin/grep -nm1 "$start"  "$phpfile.bak" | cut -d ":" -f 1`
      let "startline = $startline - 1"
      head -n $startline "$phpfile.bak" > "$phpfile"
      echo "$replace" >> "$phpfile"
      let "tail = $eof - $startline"
      tail -n $tail "$phpfile.bak" > "$phpfile.tmp"
      endline=`/usr/bin/grep -nm1 "$end" "$phpfile.tmp" | cut -d ":" -f 1`
      eof=`wc -l "$phpfile.tmp" | sed -e "s/$phpfile.tmp//"`
      let "tail = $eof - $endline"
      tail -n $tail "$phpfile.tmp" >> "$phpfile"
      rm "$phpfile.tmp"
  else
      echo "'start' not found, skipping $phpfile..."
  fi
done

---

Asuming you called it modify and did a chmod +x modify
You could use it like this..

./modify index.php otherfile.php /home/myhome/www/yetanotherfile.php

It does have a tiny bit of error handling (skip and warn if 'start' isn't found)..

I got a pm about this script giving errors when the start delimiter is found more then once in a file..
The error is in line

Code:

---

startline=`/bin/grep -n "$start"  "$phpfile.bak" | cut -d ":" -f 1`

---

A fix for this would be grep -nm1 instead of -n limiting the output to one (the first) instance..
In which case only the first instance of the start delimiter will ne used.

(edited previous post, but wanted to let you know about my mistake ;))
Bounds checking is vital.. even in scripts (especialy in scripts ??)

I'm disappointed the_JinX... :) You promised us a bash+sed script and then came up with bash only solution.

Code:

---

$ sed -i.old '/.*<div id="container">$/,/.*<\/div>$/c<?php\
  include( newstuff.php );\
?>' file1 file2 file3 ...

---

I'm not sure which platform you're using val, but I tested it on cygwin using GNU sed. It even make a backup before doing the replacement.

Peace always,
<jdenny>

Nice one, jdenny..

But mine is bigger then yours :p

For some reason that kind of logic just doesn't work on scripts ;)

I was under the assumption that sed wouldn't work on multi-line stuff, since it evaluates one line at a time..
Will have to read up on sed though... (insight welcome)..

man sed, perhaps? :D

Hehe, I do understand why that kind of logic doesn't work on scripts, as I'm a script guy myself. The author of regex is a masochist I guess... He and all who understand and use regex... :confused:

Generically, with sed, you have a command and, optionally, a range for it to work on.

So, you can break it down as "/start/,/end/command" which means, once you start seeing a line with start in it, perform command until you get a line with end in it. The command I used is c, which is:

Code:

---

      c \
      text  Replace  the  selected  lines with text, which has each embedded
              newline preceded by a backslash.

---

Doesn't tell much I know, but I've seen an example somewhere (for extracting multiple lines data from a file, using !d command).

Oh, and as for the -i option:

Code:

---

      -i[suffix], --in-place[=suffix]
              edit files in place (makes backup if extension supplied)

---

Without it, sed will displays the resulting text on standard output, so you can test it first before going "production".

Peace always,
<jdenny>

Thanks a lot !!

I really need to buy that O'Reilly's regex book.. (suggestions for other books welcome !)

Hi

As much as I understand sed (which is not much :( see below),
there is an instrinsic problem here (I think), due to the one-line/multi-line
character, as it also was mentioned by the_JinX:

We have to replace a full multi-line structure with another one.
Both script files replaces such a structure based on the delimiters,
which is dangerous. The question is, how many

Code:

---

<div id="container">

---

are there? To be safe, I have to assume a lot, because it is not
a very specific div-id. Hence, I have to make sure, that the replaced
text is in an one-to-one agreement with what I replace - the delimiters
give an idea, but it is not unique.

Many words ... :D

Quote:

---

(for extracting multiple lines data from a file, using !d command).

---

Unfortunately, I just know the basics of sed (for example I understand
your nice example jdenny). Can you also scan for multiline-patterns (not
by delimiters)?

Cheers.

Both the sed script and my (revised) bash script react to the first instance of the end delimiter after the start delimiter..

I'm not familiar enough with sed nor regular expressions to tell you eabout multiline text editing, I would think that with \n or \r\n you could just enter the whole thing..

<edit type="add">
The sed script actualy changes all instances of <div id="container"> ... </div> in any given file, mine only changes the first instance..
</edit>

Apparently you're right sec_ware... Actually I jumped in just to kid the_Jinx (and to confuse valhallen) :D

What I did is just to show that the whole bash script written by the_Jinx can be re-written in only 4 lines using sed.

The scanning for multiline patterns that I mentioned is using the same method as above (using the /start/,/end/command syntax of sed). Or did I misunderstand you?

the_Jinx, the O'Reilly's book is a must read. But I find this a decent tutorial: http://www.regular-expressions.info

Peace always,
&lt;jdenny&gt;

Hi

Jdenny, you understood me correctly (I think :D ). The delimiter method is
valid, but unfortunately a bit dangerous here (that's why years ago, I
ended up with a c program). Nevertheless, the_JinX's suggestion "\n" and/or
"\r\n" would be worth a try.

Cheers.

Note: I'm not trying to hijack val's thread... If anyone feel I need to start a new thread, please say so.

Ah, sec_ware, you meant without delimiter, I missed that (important) part. Yes, the delimiter method could be dangerous, but it's not, if you know what you're doing. I mean, it's user's responsibility to construct a valid regex to make sure that the search pattern matches what s/he wants to replace.

In this case, I'm assuming that the text between &lt;div id="container"&gt; and &lt;/div&gt; pair is unique. If it were not, then all instances which have the matching pattern would be replaced with the new text. In some cases, perhaps that's exactly what I want. But, if it turns out that the delimiter pair I choose is not unique where it should be, I need to refine my regex.

And yes, multiline regex is sed is supported, although not "straightforward". The regex code would be ugly and I think it's better to use other (better) "tools" such as Perl to do the job. Let me know if you want an example of multiline pattern in sed.

As aside note, sed is famous (notorious?) for writing ugly code. This piece of code

Code:

---

sed '
/^[why]/!d
/[do]$/!d
/^..[real]..$/!d
/^.[code,]/!d
/^..[anyway?]..$/!d
' /usr/dict/words

---

has won a “Worst Abuse of the Shell” prize at a USENIX conference few years back.

Can you guess the expected result? The author said if your copy of /usr/dict/words matches the one on his machine, you should get the following output:
hello
world

Peace always,
&lt;jdenny&gt;

I havn't got the same version (apparently) and it's not even in the same location..

Code:

---

sed '
/^[why]/!d
/[do]$/!d
/^..[real]..$/!d
/^.[code,]/!d
/^..[anyway?]..$/!d
' /usr/share/dict/words
heard
hoard

---

My bad. I don't even have the words file. I just c&p from here without testing it. I think this is the offending line:
/^..[anyway?]..$/!d

Either change !d into d , or , remove the line altogether...

Peace always,
&lt;jdenny&gt;

lol, removing that line does kinda help..

Code:

---

heard
hello
hoard
holed
world

---

using the d instead of !d is even more fun..

Code:

---

hello
holed
world

---

Ah, then insert this line after the [anyway?] line:
/^...[mate].$/d

You see, what in "holed" that doesn't exist in "hello" and "world"? I picked the fourth character, which is 'e'. Add some random characters (m, a, and t) and enclose in square brackets to make them a range of forbidden characters in 4th pos (d after the last slash is the delete command). And remember, . (dot) is represent any character (in that position).

Fun, indeed. :)

Peace always,
&lt;jdenny&gt;

yups.. Jdenny's own Hello World (tested on slackware current)..

Code:

---

sed '
/^[why]/!d
/[do]$/!d
/^..[real]..$/!d
/^.[code,]/!d
/^..[anyway?]..$/d
/^...[mate].$/d
' /usr/share/dict/words
hello
world

---