Linux basic commands

The power of combining commands
the "last" command
making the most of top
searching email with cpanel
useful bash tricks and tips
the date command -good in scripting
Oh dear file permissions are messed up
Pecl
Rpm
who are cloudflare and why should you use them
Getting more info on users
/proc/sysrq-trigger
initctl
copy and keep the same permissions
a bit more on networking
Background on Inodes
Expand on ps
Sudo
Top
Getent
How linux memory works
Logrotate
Tmux
gpg
troubleshooting swap
hostname
write
whiptail
vdir
Random password
ls
Grep searching for Ip's
scp v sftp

The power of combining commands

; - used to combine 2 commands i.e ls ;date
() – used to combine multiple commands i.e ( ls ; date ) >> new_file
| - to pass the output of 1 command to another i.e ps –ef | grep sshd

the "last" command

from the man page

Last searches back through the file /var/log/wtmp (or the file designated by the -f flag) and displays a list of all users logged in (and out) since that file was created.
last reboot will show a log of all reboots since the log file was created.
Lastb is the same as last, except that by default it shows a log of the file /var/log/btmp, which contains all the bad login attempts.

lastlog - reports the most recent login of all users or of a given user i.e lastlog
lastb shows all bad login attempts
last –x | grep –v pts shows when a server was rebooted and includes the kernel version

last -f /var/log/wtmp.1 to check another file other than /var/log/wtmp
last -R hides the hostnames field
last -F shows full login and logout times and dates
last -x shutdown searches for shutdown activity
last -x reboot searches for reboot activity

also see aulastlog and aulast

making the most of top

running top and pressing z and c will highlight the running processes, and show the full path of running processes

searching email with cpanel

In cpanel exigrep i.e exigrep "tim@bbc.co.uk" mainlog works with exim’s logs.
It finds your search string in transactions, and then gathers every log entry into separate it into complete transactions.

useful bash tricks and tips...

to re run the last command type !!
to see previous commands type history followed by !123 to rerun number 123
ctrl r abc reverse searches history for the command abc
ctrl and l to clear the page (identical to the clear command)

to add the data and time to the history timestamp run export HISTTIMEFORMAT="%F %T "
to make it permanent add it to /root/.bash_profile

the date command -good in scripting

date – shows todays date and time, i.e date, the following will show the date in the format yyyymmdd date +%Y%m%d

Oh dear file permissions are messed up

these commands are useful (and may help you out of a spot if someone has mistakanley done a recursive change permissions)

getfacl -R /path/... > /root/perms.txt # get perms of filepath
setfacl --restore=/root/perms.txt #
restore perms of filepath

Pecl

pecl install apc
PECL is a repository of PHP extensions that are made available to you via the PEAR packaging system.

Rpm

rpm does not resolve dependencies (as oppose to yum)

rpm -qa # query all
rpm -qi #find out more info
rpm -Uvh # install if it doesn't exist and update if it does
rpm -ivh #install a new kernel
rpe -e # remove

rpm -ql mod_security - to check what files/directories have been created by a process
rpm -q --whatprovides /etc/httpd/modsecurity.d/ -kind of the reverse tells you wish package created the file/directory

some other useful rpm linkshttp://www.rpm.org/max-rpm/s1-rpm-query-handy-queries.html
A link to rebuild the rpm database http://www.cyberciti.biz/tips/rebuilding-corrupted-rpm-database.html

who are cloudflare and why should you use them

cloudflare are a site who can speed up your site – DOS protection, web site stats and claim to be able to speed up your site

Getting more info on users

Finger–lookup user info i.e finger root (can be used side by side with the
id root or groups root commands)

/proc/sysrq-trigger

/proc/sysrq-trigger # you can echo commands into the file It is a 'magical' key combo you can hit which the kernel will respond to regardless of whatever else it is doing, unless it is completely locked up. C for a crash dump I to sigkill all processes w dumps blocked processes (interruptible sleep)

initctl

In unbuntu running initctl list will show you a list the contents of /etc/init

copy and keep the same permissions

cp -p keeps the same file permissions on the file the same as --preserve=mode,ownership,timestamps. This is particular important if you are say moving logs to a different location, without the -p flag the system will not be able to write to the files.

a bit more on networking

ifconfig –a displays the status of all interfaces, even those that are down

to find the status of nics on the system run service network status where:
configured devices =devices with config files in /etc/sysconfig/network-scripts
currently active devices = devices that are up (from ipconfig setup))

Background on Inodes

Inodes are assigned to every file and folder on a system, each i node is unique. Hard linked files have the same inode a softlink will not.

If you move a file it will keep the same inode if you copy a file the inode will change. (note if you move a log file any new log records can be copied over to that new file no matter what the name is until a service restart happens)

to find an inode of a device run ls -i file or stat file

if you have lots of files on a system you may (never seen it happen myself!)run out of inodes run df -i to show usage.
The easiest fix is to remove some files.

df -i will show you how many inodes are used.
find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n will show you in which folders those inodes are used.

making ps better

ps -ejH -a process tree type view (a slight twist on pstree)
ps -eLf - get info on threads as well

sudo

sudo -i and sudo -s are both useful commands to root up.

Its also worth be aware of this importance of sudoedit see http://bencane.com/2012/02/26/sudoedit-securely-allow-users-to-edit-files/

without it you can run something like

[testuser@laptop ~]$ sudo vi /etc/services
(In vi hit esc and type the following)
:!bash
[root@laptop testuser]# whoami
root

Top

In top you can save your config by pressing W while top is being run (which saves it to ~/.toprc) & then next time you run it it will use those defaults

running top and pressing z and c will highlight the running processes, and show the full path of running processes

to run top in batch mode top -b -n 1 >top.txt

some of the following is taken from http://www.linux.com/learn/tutorials/42048-uncover-the-meaning-of-tops-statistics

Expansion on what the top output means.

Processor Statistics - Tasks - Zombie: "A dead body without soul" might be a good analogy. After a child task is terminated, it is cleaned up and the only thing left is a task descriptor that includes a very important value: exit status. So if the number of zombies is high, that is a sign that one or more programs have a bug properly terminating child tasks.

Per-process Statistics -

analogue time

VIRT is like talking about the size of the land we own,
RES is the house (i.e built on top of virt -it doesn’t necessarily have to occupy the entire space)
SHR is the shared drive (built on top of virt - indicates how much of the VIRT size is actually sharable)

extra notes & more detail

VIRT : Virtual Size of the task. This includes the size of process's executable binary, the data area and all the loaded shared libraries.
RES : The size of RAM currently consumed by the task. Swapped out portion of the task is not included.
SHR : Some memory areas could be shared between two or more task, this field reflects that shared areas. The example of shared area are shared library and SysV shared memory.

A good explanation of a Wait IO troubleshooting example http://www.chileoffshore.com/en/interesting-articles/126-linux-wait-io-problem

Getent

from the man page getent - get entries from Name Service Switch libraries

it really helps you query administrative databases in Unix i.e passwd, group, hosts, services, protocols, or networks.

examples:

getent passwd root
root:x:0:0:root:/root:/bin/bash
getent hosts localhost
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

How linux memory works

A nice anolgue "In a way, Linux allocates memory the way an airline sells plane tickets. An airline will sell more tickets than they have actual seats, in the hopes that some of the passengers don't show up. Memory in Linux is managed in a similar way, but actually to a much more serious degree." (taken from http://opsmonkey.blogspot.co.uk/2007/01/linux-memory-overcommit.html)

Logrotate

As the name says it is used for rotating logs.

logrotate -dv -f /etc/logrotate.conf # will do a dry run (explanation below)

# the -d will run it in debug mode (but not actually rotate anything good for testing a new config)
# the -v will show verbosity again to help with testing/troubleshooting

If you want logrotate to run more often (for hourly log rotation, for example) you’ll need to look into using cron to run logrotate through a script in /etc/cron.hourly.
When logrotate runs it reads its configuration files to determine where to find the log files it needs to rotate, and to check on details like how often the files should be rotated and how many archived logs to keep.
/etc/logrotate.conf (defaults for log rotation)will have a line for "include /etc/logrotate.d"
ls /etc/logrotate.d Depending on how much you’ve installed on your server there may be no files in this directory, or there may be several. In general, applications that are installed through your package manager will also create a config file in /etc/logrotate.d.
/var/lib/logrotate.status is where logrotate stores information about when it last rotated a file

More reading at http://www.rackspace.com/knowledge_center/article/understanding-logrotate-utility

tmux

tux is an advanced version of screen it has this concept of windows and panes

usage

tmux new -s keywork
tmux attach -t keywork

gpg

a great tool for encrypting files

gpg- c file # for encyption (a new file called file.gpg is created)
gpg file.gpg # to decrypt

troubleshooting swap usage

rather than copy and paste someone else's content

http://northernmost.org/blog/find-out-what-is-using-your-swap/index.html

hostname

show or set the system’s host name

how to change your hostname http://www.cyberciti.biz/faq/linux-change-hostname/

write

from http://www.geekinterview.com/talk/2903-difference-between-wall-talk-write-cmds.html

Talk command is used for chatting with another online user in the same network. It opens a window where both parties can exchange messages. All users logged on the network can use this command.

Write is to send a short message to another online user. It does not keep a window open at both side as the write command do, so not suitable for continuous chatting. All users can use this also.

Wall is for sending short message not to a single user/group but all online users in the network. Only admins can use this command. Write can be blocked by each user but wall cant.

whiptail

used in coding to produce a binary output example taken from http://www.techrepublic.com/blog/linux-and-open-source/how-to-use-whiptail-to-write-interactive-shell-scripts/

put the following in a script and run it

#! /bin/bash

whiptail --yesno "Did you already know whiptail?" --yes-button "Yes, I did" --no-button "No, never heard of it" 10 70

CHOICEs=$?

vdir

similar to ls -lb

Random password

for a random password echo $(head -1 /dev/urandom | tr -dc A-Za-z0-91 | cut -c 1-8)

ls

ls -lrht # long list , human readable , order by modified time and reverse
ls -i # inode
ls -m #comma seperated list
ls -u # do not sort; list entries in directory order very good for querying a directory with thousands of files & you are getting timeouts timeouts (it removes the default alphabetical sort)

Grep searching for Ip's

to grep for an ip in a file try

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" test.txt

which will potentially find Ip's between 0.0.0.0 and 999.999.999.999. (consider it may also find numbers that aren't Ip's)

More details @ http://www.shellhacks.com/en/RegEx-Find-IP-Addresses-in-a-File-Using-Grep

scp v sftp

SCP is the abbreviation of 'secure copy', while SFTP stands for 'secure FTP'
In a nutshell, scp can only be used for transferring files, and it is non-interactive (i.e., everything has to be specified on the command line). sftp is more elaborate, and allows interactive commands to do things like creating directories, deleting directories and files (all subject to system permissions, of course),
SCP is usually much faster than SFTP at transferring files, especially on high latency networks. This happens because SCP implements a more efficient transfer algorithm, one which does not require waiting for packet confirmations. This leads to faster speed but comes at the expense of not being able to interrupt a transfer, so unlike SFTP, SCP transfer cannot be canceled without terminating the session.

the sftp command is a secure ftp
you can run various commands when you have connected i.e http://www.tecmint.com/sftp-command-examples/