Systems Administration

Here's a generic list of activities for systems' administrators - irregardless of platform or hardware (in other words - WAN admins working with Routers and Switches will find this list just as pertinant with a few adaptations as will LAN admins, server operators and P2P power users.)


===================================
Periodic sysadmin tasks:

Paramount:
--> Know what is "normal". Note what is not.



Frequent (daily?)
1. Check all systems backup logs
2. Check all systems logs
3. Check all systems security logs
4. Check all systems mail logs
5. Check all systems disk usage
6. Check for CERT and similar announcements
7. Are all machines up?
8. Check UPS status
9. Check tunnel usage report
10. Check Virus Scan rpts
--> Know what is "normal". Note what is not.

Less frequent (weekly or monthly?)
1. Verify that backups are restorable
2. Verify file system structure is valid
3. Check for undocumented system changes
4. Sniff networks for intrusion attempts
5. Check error counts on NICs
6. Check NFS errors
7. Audit router configs
8. Check for disused user-ids
9. Portscan
10. Look for weak passwords
11. Look for overly weak access controls
12. Setuid/setgid audit
13. Check available patches/service packs
14. Recall backup tapes
15. Inspect recalled tapes
16. Check for system or tool updates
17. Performance evaluation/audit
18. Printer audit/test
19. Check Conference Room / Speaker Phone operations
20. Verify system times
21. Check tape supply
22. Update documentation of network, machine configs, processess, other.

Occasional (quarterly to annually?)
1. Verify external DNSs are up and correct
2. Load test network (off hours ONLY) and baseline performance.
3. Global password changes
4. Replace tapes in backup cycles as needed.
5. Cable plant/desktop audit.
6. Update documentation that has slipped.

Quote:

---

Frequent (daily?)
1. Check all systems backup logs
2. Check all systems logs
3. Check all systems security logs
4. Check all systems mail logs
5. Check all systems disk usage
6. Check for CERT and similar announcements
7. Are all machines up?
8. Check UPS status
9. Check tunnel usage report
10. Check Virus Scan rpts

---

I'd have #7 moved to #1, hehe, right along with #8 next (battery levels, and such), and while I'm running the script that checks #1, #2, #3, #4, and #5, I'd be reading up on #6, #9, and #10. Automate everything!

Quote:

---

Less frequent (weekly or monthly?)
1. Verify that backups are restorable
2. Verify file system structure is valid
3. Check for undocumented system changes
4. Sniff networks for intrusion attempts
5. Check error counts on NICs
6. Check NFS errors
7. Audit router configs
8. Check for disused user-ids
9. Portscan
10. Look for weak passwords
11. Look for overly weak access controls
12. Setuid/setgid audit
13. Check available patches/service packs
14. Recall backup tapes
15. Inspect recalled tapes
16. Check for system or tool updates
17. Performance evaluation/audit
18. Printer audit/test
19. Check Conference Room / Speaker Phone operations
20. Verify system times
21. Check tape supply
22. Update documentation of network, machine configs, processess, other.

---

Just about everything in here, except for physical supplies can be automated into a series of scripts that I'd run every week (if not every day). Security with passwords is limited to your passwd binary and whether or not you want them to pick their own or be assigned automatically that vaguely looks like unpronounceable line-noise. Policy has a lot to do with passwords and such, whereas something arbitrary to users in dealing with setuid/gid progs is more along the lines of internal/external intrusion. Performance audits and such generally come along at the beginning of a box's inception (where they throw 9000000 users on it and expect it to fly). Documentation is superior to everything else. We set up two linux boxes with individual instructions on everything operations ran so they didn't have to remember every little thing.

Quote:

---

Occasional (quarterly to annually?)
1. Verify external DNSs are up and correct
2. Load test network (off hours ONLY) and baseline performance.
3. Global password changes
4. Replace tapes in backup cycles as needed.
5. Cable plant/desktop audit.
6. Update documentation that has slipped.

---

Hehe, if #1 is fux0red, you'll know a lot sooner than quarterly/annually. Same thing for #2, as that's something that everyone uses..although it'd be a lot better if people stoppe sending that 5 meg .mpg to 8 of their friends externally..idiots. #3 is hard to enforce because usually it stops at a manager level or is changed when a manager leaves/comes in and anything to keep the end-lusers happy is a good thing (except for us unix admins...*sigh*)

Pretty good stuff here dude, keep it up. I've noticed that a lot of these things, while should be done in certain orders (or more preferencial order), managers don't care, and neither does higher-ups. NT people have it a lot easier because they make those 18 meg excel spreadsheets and powerpoint displays that look all cutesy and ****, which is what management needs. Heaven help anyone's technical above a certain level. Bunch of lame gui-driven MS whipped fscknuts...oh wait, I'm rambling again.

Hope this helps.

Luckily the helpless (I mean Helpdesk) does most of that for me. So I can dedicate my time to more deserving pursuits.........like ...whatever.

Sounds like you guys need some lackeys.

"Helpless" ROFL dude that's funnier n' ****...I actually laughed out loud and that! Thanks for making me laugh amidst all this crap. :D

Those are good points Vorlin. Thanks for sharing your experience and giving examples of implementation of the list items.

Automation is definitely key to sanity! One must know what to automate.... what to script ... different systems and different networks will need more or fewer checks.

Honestly, and without apologies, I didn't put much thought into the "order of operations". That is going to vary with people's responsiblities and the size of the operation a person is involved with.

In a large environment or for the inexperienced person, early in their career, a person's whole life may be doing nothing but checking log files or backups or any one of these items.

I had a job once where all I did was printer maintenance. The printers ran 24/7 and I rotated through feeding them ink and paper, making maintenance adjustments, reading status reports, etc.... another time, I had a job as a tape librarian ... all I did was maintain the backup tapes ... not ever actually running backups ... maintaining the tapes ... labeling cataloguing, checking validity .... etc ...

In a smaller environment, one person may be entirely responsible for every item on that list ... and more! I've worked in those types of environments also .... :)

Thanks...although I learned what I said the hard way, by **** breaking and having to go back and get things cleaned up. As it is, I want to automate everything so then I don't have to go to work because I don't want to wear pants that day or something... (as my boss would say). :D