Printable View
Hey all,
Today I mistakenly ran rm -r on the wrong directory on a amazon cloud EBS instance - ubuntu 10.4 - ext3 file system. Shortly afterward the panic began.
Naturally we don't have backups of these files.
Is anyone aware of any tools or pointers that can help to restore these files?
I've come across http://extundelete.sourceforge.net/ but I'm uncertain if this is an appropriate solution for an amazon instance
I've used this in the past: http://www.xs4all.nl/~carlo17/howto/undelete_ext3.html
I've never used extundelete but it sounds like it is closely related to the link above if not a more maintained forked version of it.
The most important thing though would be to make sure that you stop writing anything to that filesystem.. preferably just umount instead of remount read-only before you do anything.
An EBS volume should just be able to be treated them same as any other block device so treat it like it it was directly plugged into your system and you should be fine. Are you sure you don't have any EBS snapshots? You might be able to attach one of those snapshots to your instance too.
Good luck. If ext3grep or extundelete don't get you your data then take a look at some file carvers to get as much information as you can. (I like foremost and scalpel, but there may be others out there that are better now)
No one makes snapshots? time for a board meeting.
Dude....Not having Back ups is ALWAYS a bad idea first off, and I agree that you need a board meeting to set this right. I set up a system myself where I took the VERY first Computer I EVER bought, installed a 160 GB HD in it, and then, took the Hard Drive it came with (Which is 43 GBs....Yea, 43) and then I installed Slackware Linux 12.0 on it, and set the old Hard Drive as one big partition called "/storage" and made it writable by users, and then, the new HD, I made the Root Partition, so that way if the old one craps out, I won't lose the OS and most of the Data, and I have the second HD backed up to a little External USB HD which is 80 GBs.
I set up FTP on it, and made it so that only users with valid accounts could log in, put a Hardware Firewall in front of it, and then, all the machine on the Network My Wife and I have here, can log into that box, and upload the files We need to back up. This is everything from music and movies, to software, games, and, of course, System Files. My FreeBSD machine has an FTP Service set up too, which I use as a secondary since it has more HD space than I use normally, so, I set up one on that, and use it too. Then, I set up an OLD machine, whacked VSFTPd on it, and now, I have three machines that all have an FTP Service running on them, so I can back up not only important files, but, the FreeBSD machine has two HDs in it, all for FreeBSD, and then, the Slackware FTP Server, which has SSH and all that, I can log into it with a client, upload system files I need, and then, have them stored in multiple locations. I have like 8 HDs here, and the REALLY important stuff, is saved on all of them.
The point of all that, is that basically, you could have prevented this by simply doing the following TWO things:
#1: Edit one of your Shell's Configuration files, and make rm ALWAYS do rm -i instead, so that way you can't delete anything by accident.... rm -i will ask you if you really want to delete a file, before doing it, and this can save you from what just happened. Even if you run rm -r /usr/home/username/files/* it will then ask if you're sure. And it costs nothing but 5 minutes....
#2. Set up a cheap machine with Slackware or FreeBSD, or something you're comfortable with, and put a little FTP Service on it, and then, when you're done, set up the FTP Configuration to only allow certain IPs to log in, or only allow certain users from certain IPs, and then, upload the files you need to it. If you ONLY uploaded the /etc directory and or all your configuration files, you'd barely need a gig.
I don't know anything about this sort of cloud and instance stuff, but if you can identify and access the drive, then you might try roadkil's "unstoppable copier". At the very least, clone the drive.
Also, it should be policy that if you are going to do destructive maintenance on a production system you should make and test a backup first.
Power cuts do happen, and the consequences can be very unpredictable.
@gore
43GB?.................yeah I remember that sort of thing, I actually have a couple of 123.5GB drives myself (that's what is printed on the label). I guess they all go for round numbers these days?
Yea man, it's 43 GBs heh. And yea, I think now, they go for nice round numbers now. I don't think I've seen any of those weird sizes since.
I've regularly advised this client to create backups, they have now learned but prior had difficulty following the advice. Appreciate all the advice :)
Well, most of us in this industry, or Hobby, have done it at least once. I remember once, I was getting ready to install FreeBSD 4.0 on my box, which already Dual Booted Windows 98 SE, and Linux, and was about to tri-boot with FreeBSD. I for some reason (I'm assuming lack of sleep) thought for a split second, that a good idea was to "Delete my back ups, and then re-make them AFTER the installation"..... The installer failed while doing the MBR, and I lost all my Data..... So yea, I take back ups a lot.
lololol
apt-get install safe-rm or,
alias rm=’rm -i’
power doesn't corrupt, apparently it deletes.
you're screwed dude.
Well, not totally.... I mean this is, after all, Magnetic Medium right? So actually, he DOES have the option to recover, but a few things come into play:
How long ago did it happen? I've heard some "Unix Wizard" Stories where a decently skilled Unix Wizard was able to recover deleted files by himself without resorting to the next option I'm bringing up:
Paying for Data Recovery.... I don't personally have a clue on prices for this, because I know I can't afford it. I mean to put this mildly; You'd be charged PER MB..... If you lost a 200 GB HD and a company recovers the whole thing, it's actually cheaper to have you killed for the screw up.
Another thing I know of, is back when I FIRST started in Computing, I was told "If you ever accidentally delete a file; TURN THE MACHINE OFF RIGHT AWAY! Do NOT wait for a shut down, just hit the Power Button, or pull the plug. If you're fast enough, the OS doesn't have time to write the data to disk, and if you're fast AND lucky, it'll have still been in Cache when you hit the power button, and it won't save it to disk".
Mind you, that was like 10 years ago. I got into the game pretty late lol. I've only owned a Computer for like 10 years, but even then, they were saying that you could beat the HD in speed.
There are just so many variables in this that it would take days to count them. I mean, the Operating System itself counts, as most have a way to bring something back, or "un-delete" for you DOS losers. And then the TYPE of File System matters.... Are we talking some crappy File System like FAT? Fat16? Fat32? NTFS? Ext2? Or are we talking a decent File System with a Journal like Ext3? ReiserFS? Ext4? Others? Or, are we talking FFS / UFS? ZFS? XFS? XFS was done by SGI and as far as I know, does good Journals, and Linux can use it too. BSD uses their own style of the FFS (Fast File System. Written at Berkeley) and then, Soft Updates.
I think when I was just getting started, they were actually working on this to make it better. For a time, it was one of the "selling points" that Linux VS BSD could be decided on by the File System, and it's capabilities; BSD's File System with Soft Updates, can do the whole speed thing without fragmenting the crap out of the disk, but without the loss of speed associated with it normally.
Again, your only option after this long of a time, is either pretend you had a back up and all, or, find a company who does Data Recovery, and pay out the ass for it.
I know of one company where a Magazine actually wanted to test their services and see just how good they were, so they ended up taking.... I think it was a Laptop, and they had Data on it, and actually threw it in a fire. They tossed a Laptop in a Bon Fire, and sent it in. The Company was actually able to get the Data back for them. The price of course, was well past that of most cars.
Like I said:
roadkil's "unstoppable copier"
http://www.roadkil.net/program.php?ProgramID=29
It works with Windows or Linux, and doesn't care about the file system format.
As long as the drive still spins it will go for it, even if the files/drives are damaged.
Because it works at such a low level, it takes a long time, but it will recover even parts of damaged or corrupted files without whining and crapping out like the rest of these so called recovery tools do.
The guys who recover from physically damaged or non-functional disks first try to repair the drive, then just use standard recovery tools. If they can't then they take the drive apart and try to read each platter.
OK.............you need a $250,000 spin-up table, a clean room (another $150,000) and the rest of the recovery kit and qualified technicians.
That's why they can charge what they like ;)
You might have noticed, however, that NONE of them offer to recover data from a drive that has been overwritten even once?:cool:
Just keep watching NCIS :D
Even Gutmann, the guy who came up with the..what was it...35 pass method of deletion said it was completely unnecessary. I always use one pass of pseudo random data and then 0's, and I think that's more than enough.Quote:
Originally Posted by nihil
Anyways, back on topic, this is what I've always used when people come whining to me about lost data.
http://www.pcinspector.de/default.htm?po=1&language=1
I started using it a while ago, so there may be better software now. Nihil, have you any experience with using that 'unstoppable copier' in actual tests in comparison to other software?
NCIS isn't exactly going to help with that ;) I'll give you Brits Ducky, as he's a super good guy on the show, and one of my favorite Brits, but Ziva David, is the **** on that show. Without Ziva and Gibbs and Ducky that show would suck. My Wife is biased towards Abby of course, and I do like Abby too, but MAN do I get annoyed with some of the rest of the team lol.
By the way, anyone else ever notice while watching NCIS that EVERY agent's desk contains the following machines? :
A Dell Desktop... big deal... What I really was surprised by was the fact that if you watch closely, you'll see that they also have an SGI monitor on their desks.... GOD I almost fell over the first time I noticed that there was an SGI Monitor on the desk.
The same thing happens in Jurassic Park; There are TWO scenes in that movie that I particularly talk about:
One, I always bitch about the Raptors who jump on the desk and land on the SGI machine. I don't know how many times I've screamed at my TV "GET OFF THE ****ING SGI YOU STUPID LIZARD!!!!".... And then of course, do to the sheer number of mistakes in that movie, I once looked up on the net to see if I'd missed any mistakes, because that movie has GAPING holes in it, and it's kind of sad considering who directed it, and the fact that the guy who did it actually hires people to make sure a continuation problem doesn't occur (Like a guy's watch isn't taken off by accident between scenes) and in Jurassic Park, not only is it obvious that the T-Rex scene is crap (I mean come on now, the T-Rex is in this cage, eating a SHOW animal, but a minute later, they're hanging over a CLIFF in the same spot the T-Rex was standing??? Really???) And of course at the beginning, the rich guy who is cleaning a glass with a Cloth, it changes colors every other scene, AND the Trailer they go inside to meet him, the guy opens the door from one side where the handle is located, and then you see from the other side, the next scene, the door actually opens from the OTHER side....
Mistakes like that are just funny. Like for example when the girl is sharing Ice Cream at the table when the power is out... The Ceiling Fan is spinning in that scene yet apparently the Power is out? And the guy reaches over to give her a spoon full of Ice Cream and just reaches over to give her a bite. When they pan out, the table is like 30 feet long!
Seriously how long are this guys arms?
Anyway, one of the things I found was people saying "Well the part where the little girl says "This is a Unix System! I know this!" blah blah blah that isn't Unix!" lol.... Actually it IS Unix. The 3D File Viewer the girl uses is actually an SGI Product. So that was actually not a mistake at all. SGI makes that product for IRIX.
It does say something about how amazing SGI is though; You KNOW you're a good Company when you make products so good and so futuristic that even computer people don't Believe they exist lol.
Gore, I see you wrote a tut , think it was BSD? Not sure though, will go have a look.
I am liking your idea RE: an FTP - could you maybe write a little tut for us newbies ? :)
Already did a long time ago :)Quote:
Originally Posted by Cider
http://www.antionline.com/showthread.php?t=260371
That was for SUSE and it's pretty dated, but, I think I did put the portion about configuring the text file, and in BSD, or Linux, once you've configured the actual service, you just type it like any other command to start it and you're good.
Just remember to lock the box down! You don't want people using it for other stuff ;)
And yea, the new one is for installing FreeBSD 8.2-RELEASE, and it's like my others where it's step by step instructions. I love how some of the others on other forums see it and think "This isn't very good, the normal docs are much simpler" and it's like, well, no, you just don't want an average user to be able to say they got it installed because then YOU can't say that and make it mean you're good at it lol.
http://www.antionline.com/search.php?searchid=491393
WOW have I ever written a lot of **** on here. I just searched to find the FTP one for you and forgot half of those. They're all on my server but wow.
@ metguru
Yes,Quote:
Nihil, have you any experience with using that 'unstoppable copier' in actual tests in comparison to other software?
I only use it as a last resort, or if I know that the drive is about to die, and I will probably only get one shot at it. Most regular recovery tools read the drive first and give you a list of what they can recover and how well. If the drive is dying that could in itself lose you more data? the "unstoppable copier just goes right ahead with its first pass. It is a non-destructive process as it doesn't try to rebuild fille tables or repair things..........it just copies.
Also, most recovery tools I have tried do not like damaged drives or scrambled files, and give up. The UC doesn't care ;)
It takes a hell of a long time, and requires you to have a drive or partition of the same size or bigger, as it tries to recover the lot, rather than specific files & folders.
For more mundane recoveries I generally use "Recuva" from Piriform. They are the guys who do CCleaner; their website is here:
http://www.piriform.com/recuva
"Defragler" (their defragmenter) is interesting, as it claims to defragment freespace, which I guess could be useful prior to installing stuff?
Otherwise I would use Auslogics' disk defragmenter, and take the "optimise" option. That will move most stuff to the beginning of the drive, which should, in theory, have pretty much the same effect?
:)