Monday, December 8, 2014

Command line goodies

There are actually some really good programs for the non-gui environment.Some programs can be used in multiple areas too'

User applications
  • Links2, lynx, elinks – internet
  • Bashpodder – audio podcast collector
  • Alpine – email client
  • Irssi – inter relay chat
  • Centerim – instant messaging client
  • Oleo or Sc – spreadsheet
  • Vim, emacs, nani, joe, or a dozen other programs – word processing
  • Antiword – deals with office based documents
  • Putty – secure accessing tool
  • Screen -multiple seesion tool
  • Ledger – accounting (seems to be based on gnucash)
  • gpm, mc, synaptic, sed, awk, sort, ncurses, bash ,ssh, wget, curl
  • Sqllite, mysql, psql, plus man other – databases
  • Hnb – outliner to organize ideas
  •  freebasic, gcc, python, pgp, pgp-cli, perl and etc programming language tools.
  • Too many to list here – games. (i.e ninvaders)
  • Moc, aplay, mplayer, cmis and may others – music players
  • Espeak or Festival – voice synthesizer.
  • Nget and may others – news readers.
  • Cdrecord – cd buring program.
  • wird – nice calendar tool. 
  • Dvtm  - command line windowing

Network administrator tools

ping: Check end-to-end connectivity (RTT delay, jitter, packet loss) of a remote host with ICMP echo/reply. Useful to check system status and reachability.

hping: Network scanning and testing tool that can generate ICMP/TCP/UDP ping packets. Often used for advanced port scanning, firewall testing, manual path MTU discovery and fragmentation testing.

traceroute: Discover a layer-3 forwarding path from a local host to a remote destination host with TTL-limited ICMP/UDP/TCP probe packets. Useful to troubleshoot network reachability and routing problems.

mtr: A variation of traceroute which characterizes per-hop packet loss/jitter with running statistics. Useful to characterize routing path delays.

netcat/socat: A swiss army knife of TCP/IP networking, allowing to read/write byte streams over TCP/UDP. Useful to troubleshoot firewall policies and service availability.

dig: DNS troubleshooting tool that can generate forward queries, reverse queries, find authoritative name servers, check CNAME, MX and other DNS records. Can be instructed to query a specific DNS server of your choosing.

nslookup: Another DNS checking/troubleshooting tool. Works with all DNS queries and records. Can query a particular DNS server.

dnsyo: A DNS testing tool which checks DNS propagation by performing DNS lookup from over a number of open resolvers located across 1,500 different networks around the world.

lsof: Show information about files (e.g., regular files, pipes or sockets) which are opened by processes. Useful to monitor processes or users in terms of their open network connections or opened files.

iftop: A ncurses-based TUI utility that can be used to monitor in real time bandwidth utilization and network connections for individual network interfaces. Useful to keep track of bandwidth hogging applications, users, destinations and ports.

netstat: A network statistics utility that can show status information and statistics about open network connections (TCP/UDP ports, IP addresses), routing tables, TX/RX traffic and protocols. Useful for network related diagnosis and performance tuning.

tcpdump: A popular packet sniffer tool based on libpcap packet capture library. Can define packet capturing filters in Berkeley Packet Filters format.

tshark: Another CLI packet sniffer software with full compatibility with its GUI counterpart, Wireshark. Supports 1,000 protocols and the list is growing. Useful to troubleshoot, analyze and store information on live packets.

ip: A versatile CLI networking tool which is part of iproute2 package. Used to check and modifying routing tables, network device state, and IP tunneling settings. Useful to view routing tables, add/remove static routes, configure network interfaces, and otherwise troubleshoot routing issues.

ifup/ifdown: Used to bring up or shut down a particular network interface. Often a preferred alternative to restarting the entire network service.

autossh: A program which creates an SSH session and automatically restarts the session should it disconnect. Often useful to create a persistent reverse SSH tunnel across restrictive corporate networks.

iperf: A network testing tool which measures maximum bi-directional throughput between a pair of hosts by injecting customizable TCP/UDP data streams in between.

iptables: A user-space CLI tool for configuring Linux kernel firewall. Provides means to create and modify rules for incoming, transit and outgoing packets within Linux kernel space.

nmap: A popular port scanning and network discovery tool used for security auditing purposes. Useful to find out which hosts are up and running on the local network, and what ports are open on a particular host.

TCP Wrappers: A host-based network ACL tool that can be used to filter incoming/outgoing reqeuests/replies. Often used alongside iptables as an additional layer of security.

Data management Tools

grep/egrep: Can be used to filter log content for a particular pattern or a regular expression. Variants include user-friendly ack and faster ag.

awk: A versatile text scanning and processing tool. Often used to extract certain columns or fields from text/log files, and feed the result to other tools.

sed: A text stream editor tool which can filter and transform (e.g., remove line/whitespace, substitute/convert a word, add numbering) text streams and pipeline the result to stdout/stderr or another tool.

Disk management tools.

fdisk: A disk partition editor tool. Used to view, create and modify disk partitions on hard drives and removable media.

sfdisk: A variant of fdisk which accesses or updates a partition table in a non-interactive fashion. Useful to automate disk partitioning as part of backup and recovery procedure.

parted: Another disk partition editor which can support disk larger than 2TB with GPT (GUID Partitioning Table). Gparted is a GTK+ GUI front-end of parted.

df: Used to check used/available storage and mount point of different partitions or file directories. A user-friendly variant dfc exists.

du: Used to view current disk usage associated with different files and directories (e.g., du -sh *).

mkfs: A disk formatting command used to build a filesystem on individual disk partitions. Filesystem-specific versions of mkfs exist for a number of filesystems including ext2, ext3, ext4, bfs, ntfs, vfat/fat.

fsck: A CLI tool used to check a filesystem for errors and repair where possible. Typically run automatically upon boot when necessary, but also invoked manually on demand once unmounting a partition.

mount: Used to map a physical disk partition, network share or remote storage to a local mount point. Any read/write in the mount point makes actual data being read/written in the corresponding actual storage.

mdadm: A CLI tool for managing software RAID devices on top of physical block devices. Can create, build, grow or monitor RAID array.

lvm: A suite of CLI tools for managing volume groups and physical/logical volumes, which allows one to create, resize, split and merge volumes on top of multiple physical disks with minimum downtime.
Log Processing Tools

tail: Used to monitor trailing part of a (growing) log file. Other variants include multitail (multi-window monitoring) and ztail (inotify support and regex filtering and coloring).

22. getfacl/setfacl: View and customize access control lists of files and directories, as extensions to traditional file permissions.

23. cryptsetup: Used to create and manage LUKS-encrypted disk partitions.

Backup Tools

logrotate: A CLI tool that can split, compress and mail old/large log files in a pre-defined interval. Useful for administration of busy servers which may produce a large volume of log files

rsync: A fast one-way incremental backup and mirroring tool. Often used to replicate a data repository to an offsite storage, optionally over a secure connection such as SSH or stunnel.

rdiff-backup: Another bandwidth-efficient, incremental backup tool. Maintains differential of two consecutive snapshots

Window management

screen: Used to split a single terminal into multiple persistent virtual terminals, which can also be made accessible to remote users, like teamviewer-like screen sharing.

tmux: Another terminal multiplexer tool which enables multiple persistent sessions, as well as horizontal/vertial splits of a terminal.

dvtm: windowed interface. Allows multiple windows to be used with a text terminal

Security assessment

lynis: A CLI-based vulnerability scanner tool. Can scan the entire Linux system, and report potential vulnerabilities along with possible solutions.

maldet: A malware scanner CLI tool which can detect and quarantine potentially malware-infected files. Can run as a background daemon for continuous monitoring.

rkhunter/chkrootkit: CLI tools which scan for potential rootkits, hidden backdoors and suspected exploits on a local system, and disable them.
Storage Tools

Clamav: MSWindows viros checker.

Getting help

cheat or man: A simple CLI tool which allows you to read cheat sheets of many common Linux commands, conveniently right at your fingertips. Pre-built cheat sheets are fully customizable.

apropos: Also known as man -k. Useful when you are searching man pages for descriptions or keywords.

Package Management Tools

apt: The de facto package manager for Debian based systems like Debian, Ubuntu, Backtrack or elementary OS. A life saver.

apt-fast: A supporting utility for apt-get, which can significantly improve apt-get's download speed by using multiple concurrent connections.

apt-file: Used to find out which .deb package a specific file belongs to, or to show all files in a particular .deb package. Works on both installed and non-installed packages.

dpkg: A CLI utility to install a .deb package manually. Highly advised to use apt whenever possible.

yum: The de facto automatic package manager for Red Hat based systems like RHEL, CentOS or Fedora. Yet another life saver.

pacman:  The de facto automatic package manager for Arch linux. Yet another life saver.

rpm: Typically I use rpmyum something. Has some useful parameters like -q, -f, -l for querying, files and locations, respectively.
Hardware Tools

Diagnostic tools:

lspci: A command line tool which shows various information about installed PCI devices, such as model names, device drivers, capabilities, memory address, PCI bus address.

lshw: A command line tool which queries and displays detailed information of hardware configuration in various categories (e.g., processor, memory, motherboard, network, video, storage). Supports multiple output formats: html, xml, json, text.

inxi: A comprehensive hardware reporting tool which gives an overview of various hardware components such as CPU, graphics card, sound card, network card, temperature/fan sensors, etc.

iostat: A CLI tool which reports current CPU utilization, as well as device I/O utilization, where I/O utilization (e.g., block transfer rate, byte read/write rate) is reported on a per-device or per-partition basis.

duplicity: An encrypted incremental backup utility. Uses GnuPG to encrypt a backup, and transfers to a remote server over SSH.
Performance Monitoring Tools

top: A CLI-based process viewer program. Can monitor system load, process states, CPU and memory utilization. Variants include more user-friendly htop.

ps: Shows a snapshot of all running processes in the system. The output can be customized to show PID, PPID, user, load, memory, cumulative user/system time, start time, and more. Variants include pstree which shows processes in a tree hierarchy.

nethogs: A bandwidth monitoring tool which groups active network connections by processes, and reports per-process (upload/download) bandwidth consumption in real-time.

ngxtop: A web-server access log parser and monitoring tool whose interface is inspired by top command. It can report, in real time, a sorted list of web requests along with frequency, size, HTTP return code, IP address, etc.

vmstat: A simple CLI tool which shows various run-time system properties such as process count, free memory, paging status, CPU utilization, block I/O activities, interrupt/context switch statistics, and more.

iotop: An ncurses-based I/O monitoring tool which shows in real time disk I/O activities of all running processes in sorted order.

sysdig: A versatile and comprehensive open source tool for capturing and analyzing system behavior and server state in both real-time and offline modes.
Productivity Tools

No comments:

Post a Comment