Linux Server Maintenance and Monitoring

Overview

Linux servers running services require regular maintenance and monitoring to reduce the risk of service interruptions, performance issues, and unexpected failures. This is especially important for servers that are not connected to a centralized monitoring system.

In this project, I documented a set of command-line tools and procedures for checking the health of Linux-based virtual machines, monitoring system resources, reviewing service status and logs, checking disk usage, troubleshooting network connectivity, verifying firewall settings, and managing package updates. These checks help identify potential issues and ensure that the operating system and its services are running as expected.

Objectives

• Check the operating system, virtualization platform, and system uptime.

• Monitor CPU and memory usage and identify resource-intensive processes.

• Verify service status and investigate system errors and authentication logs.

• Check disk space, partitions, memory, and swap usage.

• Review network interfaces, DNS configuration, routing tables, active connections, and listening ports.

• Verify firewall status and rules to ensure that only required services are accessible.

• Check for available package updates and perform routine package maintenance.

• Document practical commands for troubleshooting and maintaining Linux servers.

Technologies Used

• Operating Systems: Ubuntu Server, Linux

• Virtualization: VMware ESXi

• System Monitoring: htop, uptime, hostnamectl, uname, free

• Service Management: systemctl, journalctl

• Storage Management: df, lsblk

• Network Troubleshooting: ip, ifconfig, ss, systemd-resolve, Netplan

• Firewall Management: UFW

• Package Management: APT

Configurations

I use the commands and tools below to perform routine maintenance and monitor Linux-based virtual machines and the services running on them. These checks help me identify potential issues with system resources, services, storage, networking, security, and software updates.

I keep a checklist of commands with brief notes, as some commands have options that I may forget.

System

To check the machine's general status, and operating system distribution I use the following commands:

moradilab:~$ hostnamectl

   Static hostname: moradilab

      Icon name: computer-vm

        Chassis: vm

     Machine ID: 5feadd447ffe4a94a3a183e659d0c58c

        Boot ID: 28c1d21de0a147be980f3ec01e85b79a

Virtualization: vmware

  Operating System: Ubuntu 18.04.6 LTS

         Kernel: Linux 4.15.0-156-generic

   Architecture: x86-64

ali@moradilab:~$ uname -a

Linux moradilab 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Then, I check the system's uptime and load averages to get an overview of how long the machine has been running and how heavily it is being used.

Based on experience, some services may need to be restarted periodically to function properly. If necessary, I reboot the system.

:~$ uptime

 20:38:12 up 16 min,  1 user,  load average: 0.02, 0.13, 0.1

I prefer htop for monitoring system resources because its interactive interface makes it easier to review processes and resource usage than top. After running htop, I check which processes consume the most CPU and memory. I use the P and M keys, respectively, to sort processes by CPU and memory usage.

To sort processes by other criteria, such as uptime, I press F6 and select the desired sorting option.

To view the parent-child relationships between processes in a tree structure, I press F5. This helps me understand how processes are related and identify which parent process started a particular process.

Services and Users

To check whether any systemd services or other units have failed, I use the systemctl command with the --failed option.

moradilab:~$ systemctl --failed

0 loaded units listed. Pass --all to see loaded but inactive units, too.

To show all installed unit files use 'systemctl list-unit-files'.

To check the status of a specific service, including whether it is running, enabled at startup, and reporting errors, I use systemctl status.

ali@moradilab:~$ systemctl status ssh

● ssh.service - OpenBSD Secure Shell server

   Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)

   Active: active (running) since Tue 2026-10-13 20:22:17 +0330; 37min ago

  Process: 1163 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS)

 Main PID: 1181 (sshd)

Tasks: 1 (limit: 2287)

   CGroup: /system.slice/ssh.service

        └─1181 /usr/sbin/sshd -D

Oct 13 20:34:12 moradilab sshd[1719]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser=

Oct 13 20:34:15 moradilab sshd[1719]: Failed password for invalid user moradilab from 192.168.88.2 port 34080 ssh2

Oct 13 20:34:19 moradilab sshd[1719]: pam_unix(sshd:auth): check pass; user unknown

Oct 13 20:34:22 moradilab sshd[1719]: Failed password for invalid user moradilab from 192.168.88.2 port 34080 ssh2

Oct 13 20:34:30 moradilab sshd[1719]: pam_unix(sshd:auth): check pass; user unknown

Oct 13 20:34:32 moradilab sshd[1719]: Failed password for invalid user moradilab from 192.168.88.2 port 34080 ssh2

Oct 13 20:34:32 moradilab sshd[1719]: Connection reset by invalid user moradilab 192.168.88.2 port 34080 [preauth]

Oct 13 20:34:32 moradilab sshd[1719]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=19

Oct 13 20:34:57 moradilab sshd[1722]: Accepted password for ali from 192.168.88.2 port 34085 ssh2

Oct 13 20:34:57 moradilab sshd[1722]: pam_unix(sshd:session): session opened for user ali by (uid=0)

Other common systemctl commands I use to manage services include starting, stopping, restarting, and enabling them to start automatically at boot:

systemctl start [service]

systemctl stop [service]

systemctl restart [service]

systemctl enable [service]

To review system error messages recorded since the current boot, I use the following command. This helps identify problems that may have occurred after the machine started:

sudo journalctl -p err -b

To display the available journal entries in reverse chronological order, starting with the most recent, I use:

journalctl -r

To review journal entries associated with another user, I first find the user's ID using the id command. I then use journalctl to filter the logs by that user ID. This can help investigate user-specific activity when the relevant events are recorded in the system journal.

root@moradilab:/home/ali# id user-1

uid=1001(user-1) gid=1001(user-1) groups=1001(user-1)

root@moradilab:/home/ali# journalctl -uid=1001 -r

-- Logs begin at Mon 2026-10-12 18:24:48 +0330, end at Tue 2026-10-13 21:15:59 +0330. --

-- No entries --

To check disk space usage and identify filesystems that may be running out of available space, I use df -h. This command displays filesystem sizes, used space, available space, and usage percentages in a human-readable format.

moradilab:~$ df -h

Filesystem  Size  Used Avail Use% Mounted on

udev        953M 0  953M   0% /dev

tmpfs       198M  860K  197M   1% /run

/dev/sda2   3.9G  2.0G  1.8G  52% /

tmpfs       986M 0  986M   0% /dev/shm

tmpfs       5.0M 0  5.0M   0% /run/lock

tmpfs       986M 0  986M   0% /sys/fs/cgroup

/dev/sda5   2.0G  6.1M  1.9G   1% /home

/dev/sda4   2.0G   84M  1.8G   5% /boot

tmpfs       198M 0  198M   0% /run/user/1000

To view the available disks, their partitions, and their mount points, I use lsblk. This helps me understand the storage layout and identify which partitions are assigned to specific directories.

ali@moradilab:~$ lsblk

NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

sda  8:0    0  10G  0 disk

├─sda1   8:1    0   1M  0 part

├─sda2   8:2    0   4G  0 part /

├─sda3   8:3    0   2G  0 part [SWAP]

├─sda4   8:4    0   2G  0 part /boot

└─sda5   8:5    0   2G  0 part /home

To check current memory and swap usage in a concise format, I use free -h. This helps identify whether the system is under memory pressure or relying heavily on swap space.

free -h

Network

To review the network interfaces, IP addresses, and basic interface statistics, I use ifconfig or ip addr. The ip command is the preferred modern option and is generally available on Linux distributions, while ifconfig may require installing the package.

ali@moradilab:~$ ifconfig

ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

     inet 192.168.88.10  netmask 255.255.255.0  broadcast 192.168.88.255

     inet6 fe80::20c:29ff:fe86:438a  prefixlen 64  scopeid 0x20<link>

     ether 00:0c:29:86:43:8a  txqueuelen 1000  (Ethernet)

     RX packets 160566  bytes 213294172 (213.2 MB)

     RX errors 0  dropped 1745  overruns 0  frame 0

     TX packets 86725  bytes 6319331 (6.3 MB)

     TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

     inet 127.0.0.1  netmask 255.0.0.0

     inet6 ::1  prefixlen 128  scopeid 0x10<host>

     loop  txqueuelen 1000  (Local Loopback)

     RX packets 118  bytes 10622 (10.6 KB)

     RX errors 0  dropped 0  overruns 0  frame 0

     TX packets 118  bytes 10622 (10.6 KB)

     TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ali@moradilab:~$ ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

    valid_lft forever preferred_lft forever

inet6 ::1/128 scope host

    valid_lft forever preferred_lft forever

2: ens35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000

link/ether 00:0c:29:86:43:8a brd ff:ff:ff:ff:ff:ff

inet 192.168.88.10/24 brd 192.168.88.255 scope global ens35

    valid_lft forever preferred_lft forever

inet6 fe80::20c:29ff:fe86:438a/64 scope link

    valid_lft forever preferred_lft forever

To check the DNS servers and search domains configured on the machine, I use systemd-resolve --status. This helps verify the DNS configuration when troubleshooting name resolution issues.

ali@moradilab:~$ systemd-resolve --status

       DNSSEC NTA: 10.in-addr.arpa

   Current Scopes: DNS

MulticastDNS setting: no

   DNSSEC setting: no

DNSSEC supported: no

      DNS Servers: 192.168.88.1

       DNS Domain: mike.rt

To add or change a DNS server in a Netplan configuration, I edit the relevant YAML file.

ali@moradilab:~$ sudo nano /etc/netplan/00-installer-config.yaml

After saving the changes to the YAML file, I use netplan try to test the new network configuration. This command allows me to confirm the changes before they are permanently applied and can automatically roll back the configuration if it is not confirmed.

ali@moradilab:~$ sudo netplan try

To review the routing table and identify the default gateway and routes available to the machine, I use ip route.

ali@moradilab:~$ ip route

default via 192.168.88.1 dev ens35 proto static

192.168.88.0/24 dev ens35 proto kernel scope link src 192.168.88.10

To review active network connections and the ports they use, I run ss -tunap. This displays TCP and UDP sockets, numeric addresses and ports, and associated processes. It helps identify active connections and the applications using network resources.

ali@moradilab:~$ ss -tunap

Netid  State     Recv-Q    Send-Q            Local Address:Port           Peer Address:Port

udp    UNCONN    0         0                 127.0.0.53%lo:53                  0.0.0.0:*

tcp    LISTEN    0         128               127.0.0.53%lo:53                  0.0.0.0:*

tcp    LISTEN    0         128                     0.0.0.0:22                  0.0.0.0:*

tcp    ESTAB     0         0                 192.168.88.10:22             192.168.88.2:34085

tcp    LISTEN    0         128                        [::]:22                     [::]:*

To check listening TCP and UDP ports, I use ss -tulnp. This helps identify services waiting for incoming connections and verify whether any unexpected services are listening on network interfaces.

ali@moradilab:~$ ss -tulnp

Netid   State     Recv-Q    Send-Q            Local Address:Port           Peer Address:Port

udp     UNCONN    0         0                 127.0.0.53%lo:53                  0.0.0.0:*

tcp     LISTEN    0         128               127.0.0.53%lo:53                  0.0.0.0:*

tcp     LISTEN    0         128                     0.0.0.0:22                  0.0.0.0:*

tcp     LISTEN    0         128 

To check ssh authentication logs:

ali@moradilab:~$ journalctl -u ssh -r

-- Logs begin at Mon 2026-10-12 18:24:48 +0330, end at Tue 2026-10-13 23:21:27 +0330. --

Oct 13 23:21:27 moradilab sshd[1302]: pam_unix(sshd:session): session opened for user ali by (uid=0)

Oct 13 23:21:27 moradilab sshd[1302]: Accepted password for ali from 192.168.88.2 port 29447 ssh2

Oct 13 23:19:29 moradilab systemd[1]: Started OpenBSD Secure Shell server.

Oct 13 23:19:29 moradilab sshd[1207]: Server listening on :: port 22.

Oct 13 23:19:29 moradilab sshd[1207]: Server listening on 0.0.0.0 port 22.

Oct 13 23:19:29 moradilab systemd[1]: Starting OpenBSD Secure Shell server...

-- Reboot –

Check firewall status:

root@moradilab:/home/ali# ufw status

Status: active

To                     Action  From

--                     ------  ----

OpenSSH                ALLOW   Anywhere

OpenSSH (v6)           ALLOW   Anywhere (v6)

root@moradilab:/home/ali# ufw status verbose

Status: active

Logging: on (low)

Default: deny (incoming), allow (outgoing), disabled (routed)

New profiles: skip

To                     Action  From

--                     ------  ----

22/tcp (OpenSSH)       ALLOW IN Anywhere

22/tcp (OpenSSH (v6))  ALLOW IN Anywhere (v6)

If firewall is not enabled and it is needed to be enabled:

sudo ufw enable

To see rules added to the firewall manually:

root@moradilab:/home/ali# ufw show added

Added user rules (see 'ufw status' for running firewall):

ufw allow OpenSSH

In case I need to allow a port and protocol, such as https webadmin:

sudo ufw allow 443/TCP

Update

As part of periodic maintenance, I check for available package updates and install them when appropriate. Regular updates help address security vulnerabilities, fix software bugs, and maintain system stability. I review the available updates and consider their potential impact before applying them to production systems.

Update package list and install available updates

apt list –upgradable

upgrade one package

security updates auto install

apt autoclean and autoremove