RHEL 6 Notes: October 2012

Oct 12, 2012

Configure Software Raid in Linux

The term RAID is an acronym for the phrase, Redundant Array of Independent Disks. RAID is a way of combining the storage available across multiple disks and supplying users a single, unified virtual device.
RAID can be used to provide:

data integrity
fault tolerance
improved performance
greater storage capacity

Configuring Software RAID

Configuring RAID using Linux requires a number of steps that need to be followed carefully. In the tutorial example, we will be configuring RAID 5 using a system with three pre-partitioned hard disks. The partitions to be used are:

/dev/hde1

/dev/hdf2
/dev/hdg1

RAID Partitioning

You first need to identify two or more partitions, each on a separate disk. If you are doing RAID 0 or RAID 5, the partitions should be of approximately the same size, as in this scenario. RAID limits the extent of data access on each partition to an area no larger than that of the smallest partition in the RAID set.

Determining Available Partitions

First use the fdisk -l command to view all the mounted and unmounted filesystems available on your system. You may then also want to use the df -k command, which shows only mounted filesystems but has the big advantage of giving you the mount points too.

These two commands should help you to easily identify the partitions you want to use. Here is some sample output of these commands.

[root@linuxbox tmp]# fdisk -l

Disk /dev/hda: 12.0 GB, 12072517632 bytes
255 heads, 63 sectors/track, 1467 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot	Start	End	Blocks	Id	System
/dev/hda1	1	13	104391	83	Linux
/dev/hda2	14	144	1052257+	83	Linux
/dev/hda3	145	209	522112+	82	Linux swap
/dev/hda4	210	1467	10104885	5	Extended
/dev/hda5	210	655	3582463+	83	Linux
...
...
/dev/hda15	1455	1467	104391
[root@linuxbox tmp# df -k
Filesystem	1K-blocks	Used	Available	Use%	Mounted on
/dev/hda2	1035692	163916	819164	17%	/
/dev/hda1	101086	8357	87510	9%	/boot
/dev/hda15	101086	4127	91740	5%	/data1
...
...
...
/dev/hda7	5336664	464228	4601344	10%	/var

Unmount the Partitions

You don't want anyone else accessing these partitions while you are creating the RAID set, so you need to make sure they are unmounted.

[root@linuxbox tmp]# umount /dev/hde1
[root@linuxbox tmp]# umount /dev/hdf2
[root@linuxbox tmp]# umount /dev/hdg1

Prepare The Partitions With FDISK
You have to change each partition in the RAID set to be of type FD (Linux raid autodetect), and you can do this with fdisk. Here is an example using /dev/hde1.

[root@linuxbox tmp]# fdisk /dev/hde
The number of cylinders for this disk is set to 8355.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help):

Use FDISK Help

Now use the fdisk m command to get some help:
Command (m for help): m
...
...
p   print the partition table
q   quit without saving changes
s   create a new empty Sun disklabel
t   change a partition's system id
...
...
Command (m for help):

RAID in Linux & RAID Levels

RAID in Linux

data integrity
fault tolerance
improved performance
greater storage capacity

Hard disks are mechanical devices involving moving parts and unfortunately tend to fail over time. There are also physical limits to the speed at which data can be read and/or written to disks. RAID helps mitigate this risk by protecting data stored on hard disks and improving disk performance by writing the data to multiple

Physical locations according to several different schemas, known as "RAID Levels". Furthermore, RAID can be provided by either dedicated, specialized hardware or by the operating system at a virtual layer.

Hardware RAID solutions exist that operate as dedicated devices, usually as PCI expansion cards or directly on the motherboard. The independent disks attach to the hardware interface. In a true hardware RAID, the operating system simply writes data to the hardware RAID controller which handles the multiplicitous reads and writes to the associated disks. Other so−called hardware RAIDs rely on special drivers to the operating system; these act more like software RAIDs in practice. With current technology, hardware RAIDconfigurations are generally chosen for very large RAIDs.

Additionally, some operating systems, including Linux®, provide RAID functionality within a software layer.RAID partitions are logically combined and a virtual device appears to higher layers of the operating system in place of the multiple constituent devices. This solution is often a high−performance and inexpensive alternative available for RAID users.

RAID levels

There are many RAID levels. It will be impossible to list them all here. In this classroom, we will mention the most common, most important RAID types, all of which are fully supported by Linux.

RAID 0 (Striping)

This level is achieved by grouping 2 or more hard disks into a single unit with the total size equaling that of all disks used.

Practical example: 3 disks, each 80GB in size can be used in a 240GB RAID 0 configuration.

RAID 0 works by breaking data into fragments and writing to all disk simultaneously. This significantly improves the read and write performance.

On the other hand, no single disk contains the entire information for any bit of data committed. This means that if one of the disks fails, the entire RAID is rendered inoperable, with unrecoverable loss of data.

RAID 0 is suitable for non-critical operations that require good performance, like the system partition or the /tmp partition where lots of temporary data is constantly written. It is not suitable for data storage.

Uasable Space in Raid level0 = (smallest disk) * (no. of disks)

Kernel in Linux, kernel services & configurations

Kernel :- kernel is the heart of the operating system. It manages communication with hardware, decides which processes to run, and provides each process with an isolated, virtual address space in which to run. The kernel is what the GRUB boot loader loads into memory. The kernel loads device driver modules. It also allocates hardware resources such as IRQ ports, I/O addresses, and DMA channels. A recompiled kernel can lead to:

Greatly improved speed at which kernel services operate.
Direct support for commonly used drivers.
Dynamic loading of appropriate drivers as modules.
Lower memory consumption by removing unneeded components.
Support for high-end hardware, such as memory above 4GB,hardware array controllers, symmetric multiprocessing (multiple CPU) support, and more.

Kernels can be organized as one big unit or as a lot of interconnected pieces. Kernels are called up by boot loaders when you start your system.

Monolithic Versus Modular

A monolithic kernel is a kernel in which all the device modules are built directly into the kernel.

Modular kernels have many of their devices built as separate loadable modules. Monolithic kernels can communicate with devices faster, since the kernels can talk to the hardware only indirectly through a module table. Unfortunately, the typical monolithic kernel is huge, which reduces available RAM. In addition, some systems just can't boot a kernel that's too large.

Linux once had problems loading modular kernels for some hardware. With a monolithic kernel, the drivers are already there and are often more appropriate for certain components such as embedded hardware.

A modular kernel has greater flexibility. We can compile almost all drivers as modules, and then each module can be inserted into the kernel whenever you need it. Modules keep the initial kernel size low, which decreases the boot time and improves overall performance. If Linux has trouble loading a kernel module, We can use the modprobe or insmod command to load modules as needed, and add those options to the /etc/modprobe.conf file.(in Centos5/Rhel5).

Upgrading Kernel :-

Updating the kernel is not as difficult as it looks. We should never overwrite or upgrade an existing kernel, as mistakes happen. New kernels are handled by installing the newly built kernel in /boot and then adding another boot option to your boot loader configuration file (/boot/grub/grub.conf) for the new kernel. GRUB treats the new kernel as if it were an entirely new operating system. If we install the new kernel directly from a Red Hat configured RPM, it updates the boot loader automatically.

If you do make a drastic mistake and the kernel doesn't boot, you can simply reboot the system and select the old kernel from the GRUB menu.

There are different versions of kernel-devel, kernel-PAE, kernel-xen, and kernel-headers packages for each supported architecture.

Available Linux Kernels (and Related Packages)

Kernel RPM	Description / Architecture
kernel-versionnum.i686	Designed for PCs with a single Intel/AMD CPU; also works with dual-core systems
kernel-versionnum.ia64	Designed for Itanium2 systems
kernel-devel-versionnum	Installs drivers and other information to help compile third-party drivers
kernel-PAE-versionnum	If you have more than 4GB of RAM, install the PAE kernel associated with your CPU architecture
kernel-PAE-devel-versionnum	If you have more than 4GB of RAM, install the PAE kernel associated with your CPU architecture
kernel-headers-versionnum	Includes kernel headers; often sufficient for drivers
kernel-versionnum.src.rpm	Includes the source code for the RHEL kernel

How to compile a new kernel in Linux (Centos-5.3)

when we purchased a new hardware but the current kernel does not support it, or need to add functionality to the kernel that does not come from the factory.

It is at those times that is necessary to compile a new kernel for the device to be recognized or a new functionality can be used by the software.

Step:1 Install the required packgaes for compling the new kernel

# yum install gcc make bison ncurses-devel rpm-build

Step:2 Download the latest Kernel using this url “ http://www.kernel.org/pub/linux/kernel/ ” , in my case I am using linux-2.6.36.2.tar.bz2" package .

# bunzip2 linux-2.6.36.2.tar.bz2
# tar xvf linux-2.6.36.2.tar

Step:3 Now got to extracted directory of latest kernel

#cd linux-2.6.36.2
Copy Kernel source code directory "linux-2.6.36.2"in "/usr/src" directory,

#cp -r linux-2.6.36.2 /usr/src/

Step:4 Go to "/usr/src/linux-2.6.36.2/" and run,

#make menuconfig

This will open Linux Kernel configuration window for you to select & load extra modules. Make selection as below, you can choose something different .

Step:5 Select modules whatever you want, Click "Enter" to go to sub menu, & press "Space" to enable , load & disable modules.

In my case i am enabling NTFS file system in Linux, to make it enable do like this

Disk Partitioning & Managing Partitions

Disk Partitioning & Managing Partitions

When we format a computer hard drive, we will lose everything that is on the drive. Therefore, it is very important to back up anything you might later want.To format a secondary drive, we need root access. Linux allows only 4 primary partitions.

On an IDE drive, the first drive is called hda, and the partitions are shown as hda1, hda2 . . . . etc. etc. Your second drive is called hdb.On an IDE drive you can have up to 63 partitions, 3 primary and 60 logical ( contained in one extended partition )

On a SCSI drive, the first drive is called sda, the partitions are sda1, sda2 . . The second drive is called sdb.On an IDE drive you can have up to 63 partitions, 3 primary and 60 logical ( contained in one extended partition )

An extended partition is the only kind of partition that can have multiple partitions inside. Think of it like a box that contains other boxes, the logical partitions. The extended partition can't store anything, it's just a holder for logical partitions.

The extended partitions is a way to get around the fact you can only have four primary partitions on a drive. You can put lots of logical partitions inside it.

We can see all the drives attached to your system by typing the command "ls /dev/hd*" or "ls /dev/sd*", depending on which type (IDE, SATA and so forth) the drives are. On the example system, the result of this command looks like "/dev/hda /dev/hda1 /dev/hda2 /dev/hdb /dev/hdb1". The operating system is installed on hda, which has two partitions (hda1 and hda2), and there is one partition on hdb and hdb1.

Steps for Creating partition using fdisk command : -

Step:1 To list available drives on you machine type:

# fdisk -l

Choose the drive you want to make changes to and engage it using fdisk:

# fdisk /dev/sdc

replace the "sdc" with the drive you want to edit.

The basic fdisk commands you need are:

m - print help

p - print the partition table

n - create a new partition

d - delete a partition

q - quit without saving changes

w - write the new partition table and exit

Step:2 Enter "p" to see the partition table of the drive. The first line of output from the "p" command will also tell you the size of the drive. This is a good way to double-check that you are working with the correct drive.

Step:3 Type "n" and hit "Enter." Then press "p" to create a primary partition. It asks you for a partition number; enter "1." Now you are asked which cylinder the partition should start at. The beginning of the drive is the default, so just hit "Enter." Then, you are asked for the last cylinder. The end of the drive is the default, so you can just press "Enter" again.

Step:4 Now you are back at fdisk's command prompt. Use the "p" command to check the partition table. You should now see your new partition at the bottom of the output.

Step:5 Now we need to set the filesystem type for your new partition with the "t" command. We are asked for the Hex code of the filesystem you wish to use. We will use the standard Linux ext2 filesystem, which is "83." If you are doing something special and know of a particular filesystem that you need to use, you can press "L" to see all the codes, which are one or two characters made up of the numbers 0 to 9 and the letters a to f.

Process and Daemon Process in Linux

Process :- Process can be defined as “ Any program in execution is called processs”

Daemon Process :- A daemon process is a program that is run in the background and provide some system services. On a Linux system, some processes start at boot time and continue running until the system is powered down or until you execute a kill command. These processes typically provide some specific function such as serving pages , sharing print queue or processing loging requests. In Unix World these types of processes are called daemons.

A daemon process runs in the background , waiting for events to happen , When the daemon receives a request , typically it forks a copy of itself to process the request. Meanwhile the parent goes back to wait for the next event or request.

The Apache Webserver is an example of a daemon, It starts running when the system first boots, and remains in the memory servicing requests for web pages until the system is shutdown or “httpd”
daemon is manually stopped.

Types of Daemons :-

Standalone
Transient (Controlled by the super daemon xinetd)

Daemons may be started up in a number of ways. “standalone” daemons are programs that run all the time & are started up either bu init (such as mingetty and gdm) or by a startup script under /etc/rc.d. “Transient” daemons are only started up when they are needed,& are controlled by a standalone daemon called xinetd. Standalone daemons can be used more quickly, but take up the space in the memory and cpu cycles all the time, whether or not they are actively being used.

Typically a daemon process is not connected to a TTY, so is shown in a ps report with a TTY of '?'.From this , you may also recognize processes which are managed by daemons, as they too will show a '?' in the TTY field

Process States :-

Running: This is a state where a process is either in running or ready to run.

Interruptible: This state is a blocked state of a process which awaits for an event or a signal from another process

Uninterruptible: It is also a blocked state. The process is forced to halt for certain condition that a hardware status is waited and a signal could not be handled.

Stopped: Once the process is completed, this state occurs. This process can be restarted

Zombie or Defunct process : Just before a process dies ,it notifies its parent & waits for acknowledgement .If the parent process does not acknowledge this notification,all the resources except for the PID are released . Zombie Process are cleared from the system during the next system reboot.

Command to see the zombie process :

[root@mail7 ~]# ps -ef
UID	PID	PPID	C	STIME	TTY	TIME	CMD
root	1	0	0	May10	?	00:00:00	init [3]
root	2	1	0	May10	?	00:00:12	[migration/0]
root	3	1	0	May10	?	00:00:00	[ksoftirqd/0]
root	4	1	0	May10	?	00:00:00	[watchdog/0]
root	5	1	0	May10	?	00:00:00	[events/0]

Where:

-e to display all the processes.
-f to display full format listing.

Internet Services in Linux

Internet Services :

An internet Service can be defined as any service that can be accessed through TCP/IP based netwroks , whether an internal network(Intranet) or external network(Internet).Actually TCP & IP two protocols that are included in a group of protocols sometimes knows as the Internet Protocols. Internet Services can can be provided through either secure or non secure TCP/IP connections. Common services are Telnet,FTP,SMTP,HTTP,ICMP,ARP,DNS,ssh,scp,sftp & others.

TCP/IP provides a platform and operating system independent protocols for these services. Any computer , running any operating system can communicate with any other network on the network if they both use TCP/IP protocols for establishing & maintaining the connection and formating & transmitting the data.

Secure Services

Common services such as Telnet and FTP were written in the days when every one trusted every body else on the Internet. These services send all of their traffic in plain text . Including passwords.

Plain text is extremely easy to eavesdrop on by anyone between the traffic's source and destination. Since the Internet has exploded in popularity , running insecure services such these is not a good idea. That's why secure replacements have been developed. These replacements provide stronger authentication controls & encrypt all their traffic to keep your data safe. We should always run secure services instead of insecure services.

The availability of a wide range of Internet Services makes Linux machine versatile workhorses that can fullfill many different functions in a company's network.

Ssh :

Secure shell also known as ssh , is a secure telnet replacements that encrypts all traffic ,including passwords , using a public/private encryption key exchange protocols . It provides the same functionality of Telnet , plus other useful functions such as traffic tunneling.

Ssh asks you if you want to accept and trust the host key being sent to you as being the real key. This questtions is asked only once , when you login in to a machine for the first time. After this first login , ssh behaves exactly like telnet - you start ssh , it asks for your password as shown in above diagram , then you have the regular terminal screen .

In addition to provide terminal access , ssh tunnels almost any other protocols through it. So it is possible to tunnel POP,rcp, and other protocols through ssh to turn them into encrypted , more secure protocols. With enough imagination and practice , you can make almost any thing more secure with ssh. Following is an example of how to tunnel your mail through ssh in order to keep your password and mail encrypted & secure during transit.

In this example We use pop3 to retrieve our mail from the remote machine buffy.xena.edu. Normally we would tell our pop3 software to connect from your localhost to port 110(pop3 port) of buffy.xena.edu server.

But in this example first step is to configure your pop mailer to connect to port 16510 of our own machine and put in the password for our account on buffy.xena.edu. The second step is to set up the ssh tunnel , which encrypts and forward the traffic over the network to buffy.xena.edu's pop port.

To setup the ssh tunnel , type the below command

# ssh -N -L 16510:127.0.0.1:110 vnarat@buffy.xena.edu

Now you can send and receive your mails through an encrypted ssh tunnel.

RHEL 6 Notes