Name: Sysadmin
Availability: InStock
Author: ivangdavila

System Administration Rules

Create service accounts with
```
--system
```
flag — no home directory, no login shell
```
sudo
```
with specific commands, not blanket ALL — principle of least privilege
Lock accounts instead of deleting:
```
usermod -L
```
— preserves audit trail and file ownership
SSH keys in
```
~/.ssh/authorized_keys
```
with restrictive permissions — 600 for file, 700 for directory
```
visudo
```
to edit sudoers — catches syntax errors before saving, prevents lockout

```
systemctl
```
for services, not
```
service
```
— systemd is standard on modern distros
```
journalctl -u service -f
```
for live logs — more powerful than tail on log files
```
nice
```
and
```
ionice
```
for background tasks — don't compete with production workloads
Kill signals: SIGTERM (15) first, SIGKILL (9) last resort — SIGKILL doesn't allow cleanup
```
nohup
```
or
```
screen
```
/
```
tmux
```
for long-running commands — SSH disconnect kills regular processes

```
df -h
```
for disk usage,
```
du -sh *
```
to find culprits — check before disk fills completely
```
lsof +D /path
```
finds processes using a directory — needed before unmounting
```
ncdu
```
for interactive disk usage — faster than repeated du commands
Mount options matter:
```
noexec
```
,
```
nosuid
```
for security on data partitions
Resize filesystems with care: grow is safe, shrink risks data loss — always backup first

```
chmod 600
```
for secrets,
```
640
```
for configs,
```
644
```
for public — world-writable is almost never correct
Sticky bit on shared directories (
```
chmod +t
```
) — users can only delete their own files
```
setfacl
```
for complex permissions — when traditional owner/group/other isn't enough
```
chattr +i
```
makes files immutable — even root can't modify without removing flag
SELinux/AppArmor in enforcing mode — permissive logs but doesn't protect

```
apt update
```
before
```
apt upgrade
```
— upgrade without update uses stale package lists
Unattended security updates:
```
unattended-upgrades
```
— critical patches shouldn't wait
Pin package versions in production — unexpected upgrades cause unexpected outages
Remove unused packages:
```
apt autoremove
```
— reduces attack surface and disk usage
Know your package manager: apt/yum/dnf/pacman — commands differ, concepts similar

Test restores regularly — backups that can't restore are worthless
Include package lists and configs, not just data — recreating environment is painful
Offsite backups mandatory — local backups don't survive disk failure or ransomware
Backup before any risky change — "I'll just quickly edit" famous last words
Document restore procedure — 3am disaster is wrong time to figure it out

```
top
```
/
```
htop
```
for live view,
```
vmstat
```
for trends — understand baseline before diagnosing
```
iotop
```
for disk I/O bottlenecks — slow disk often blamed on CPU
Load average: 1.0 per core is healthy — consistently higher means queuing
Swap usage isn't inherently bad — but consistent swapping indicates memory shortage
```
sar
```
for historical data — retroactively diagnose what happened during incident

```
ss -tulpn
```
shows listening ports —
```
netstat
```
is deprecated
```
ip addr
```
and
```
ip route
```
replace
```
ifconfig
```
and
```
route
```
— learn the new tools
Check both host firewall and cloud security groups — traffic blocked at either level fails
```
/etc/hosts
```
for local overrides — quick testing without DNS changes
```
curl -v
```
shows full connection details — headers, timing, TLS handshake

Sysadmin